# THE MECHANISMS OF INSECT COGNITION

EDITED BY : Martin Giurfa, Jeffrey A. Riffell and Lars Chittka PUBLISHED IN : Frontiers in Psychology

#### Frontiers eBook Copyright Statement

The copyright in the text of individual articles in this eBook is the property of their respective authors or their respective institutions or funders. The copyright in graphics and images within each article may be subject to copyright of other parties. In both cases this is subject to a license granted to Frontiers. The compilation of articles constituting this eBook is the property of Frontiers.

Each article within this eBook, and the eBook itself, are published under the most recent version of the Creative Commons CC-BY licence. The version current at the date of publication of this eBook is CC-BY 4.0. If the CC-BY licence is updated, the licence granted by Frontiers is automatically updated to the new version.

When exercising any right under the CC-BY licence, Frontiers must be attributed as the original publisher of the article or eBook, as applicable.

Authors have the responsibility of ensuring that any graphics or other materials which are the property of others may be included in the CC-BY licence, but this should be checked before relying on the CC-BY licence to reproduce those materials. Any copyright notices relating to those materials must be complied with.

Copyright and source acknowledgement notices may not be removed and must be displayed in any copy, derivative work or partial copy which includes the elements in question.

All copyright, and all rights therein, are protected by national and international copyright laws. The above represents a summary only. For further information please read Frontiers' Conditions for Website Use and Copyright Statement, and the applicable CC-BY licence.

ISSN 1664-8714 ISBN 978-2-88963-490-3 DOI 10.3389/978-2-88963-490-3

#### About Frontiers

Frontiers is more than just an open-access publisher of scholarly articles: it is a pioneering approach to the world of academia, radically improving the way scholarly research is managed. The grand vision of Frontiers is a world where all people have an equal opportunity to seek, share and generate knowledge. Frontiers provides immediate and permanent online open access to all its publications, but this alone is not enough to realize our grand goals.

#### Frontiers Journal Series

The Frontiers Journal Series is a multi-tier and interdisciplinary set of open-access, online journals, promising a paradigm shift from the current review, selection and dissemination processes in academic publishing. All Frontiers journals are driven by researchers for researchers; therefore, they constitute a service to the scholarly community. At the same time, the Frontiers Journal Series operates on a revolutionary invention, the tiered publishing system, initially addressing specific communities of scholars, and gradually climbing up to broader public understanding, thus serving the interests of the lay society, too.

#### Dedication to Quality

Each Frontiers article is a landmark of the highest quality, thanks to genuinely collaborative interactions between authors and review editors, who include some of the world's best academicians. Research must be certified by peers before entering a stream of knowledge that may eventually reach the public - and shape society; therefore, Frontiers only applies the most rigorous and unbiased reviews.

Frontiers revolutionizes research publishing by freely delivering the most outstanding research, evaluated with no bias from both the academic and social point of view. By applying the most advanced information technologies, Frontiers is catapulting scholarly publishing into a new generation.

#### What are Frontiers Research Topics?

Frontiers Research Topics are very popular trademarks of the Frontiers Journals Series: they are collections of at least ten articles, all centered on a particular subject. With their unique mix of varied contributions from Original Research to Review Articles, Frontiers Research Topics unify the most influential researchers, the latest key findings and historical advances in a hot research area! Find out more on how to host your own Frontiers Research Topic or contribute to one as an author by contacting the Frontiers Editorial Office: researchtopics@frontiersin.org

Frontiers in Psychology 1 February 2020 | The Mechanisms of Insect Cognition

# THE MECHANISMS OF INSECT COGNITION

Topic Editors:

Martin Giurfa, Centre de Recherches sur la Cognition Animale - Centre de Biologie Intégrative, Université Paul Sabatier Toulouse III, CNRS, France Jeffrey A. Riffell, University of Washington, United States Lars Chittka, Queen Mary University of London, United Kingdom

Image: Yuval Vaknin/Shutterstock.com

Citation: Giurfa, M., Riffell, J. A., Chittka, L., eds. (2020).The Mechanisms of Insect Cognition. Lausanne: Frontiers Media SA. doi: 10.3389/978-2-88963-490-3

# Table of Contents

*05 Editorial: The Mechanisms of Insect Cognition* Lars Chittka, Martin Giurfa and Jeffrey A. Riffell *08 Behavioral Evidence for Enhanced Processing of the Minor Component of Binary Odor Mixtures in Larval* Drosophila Yi-chun Chen, Dushyant Mishra, Sebastian Gläß and Bertram Gerber *16 The View From the Trees: Nocturnal Bull Ants,* Myrmecia midas*, Use the Surrounding Panorama While Descending From Trees* Cody A. Freas, Antione Wystrach, Ajay Narendra and Ken Cheng *31 Lateralization of Sucrose Responsiveness and Non-associative Learning in Honeybees* David Baracchi, Elisa Rigosi, Gabriela de Brito Sanchez and Martin Giurfa *40 Odor Learning and Its Experience-Dependent Modulation in the South American Native Bumblebee* Bombus atratus *(Hymenoptera: Apidae)* Florencia Palottini, María C. Estravis Barcala and Walter M. Farina *50 Mind Control: How Parasites Manipulate Cognitive Functions in Their Insect Hosts* Frederic Libersat, Maayan Kaiser and Stav Emanuel *56 How to Navigate in Different Environments and Situations: Lessons From Ants* Cody A. Freas and Patrick Schultheiss *63 Cognitive Aspects of Comb-Building in the Honeybee?* Vincent Gallo and Lars Chittka *72* Egr-1*: A Candidate Transcription Factor Involved in Molecular Processes Underlying Time-Memory* Aridni Shah, Rikesh Jain and Axel Brockmann *84 Omega-6:3 Ratio More Than Absolute Lipid Level in Diet Affects Associative Learning in Honey Bees* Yael Arien, Arnon Dag and Sharoni Shafir *92 Maggot Instructor: Semi-Automated Analysis of Learning and Memory in*  Drosophila *Larvae* Urte Tomasiunaite, Annekathrin Widmann and Andreas S. Thum *110 The Drivers of Heuristic Optimization in Insect Object Manufacture and Use* Natasha Mhatre and Daniel Robert *121 Signaling Pathways for Long-Term Memory Formation in the Cricket* Yukihisa Matsumoto, Chihiro S. Matsumoto and Makoto Mizunami *129 Learning Spatial Aversion is Sensory-Specific in the Hematophagous Insect* Rhodnius prolixus Sebastian Minoli, Agustina Cano, Gina Pontes, Amorina Magallanes, Nahuel Roldán and Romina B. Barrozo *140 Application of a Prediction Error Theory to Pavlovian Conditioning in an Insect*

Makoto Mizunami, Kanta Terao and Beatriz Alvarez

*148 Multiple Representations of Space by the Cockroach,* Periplaneta americana

Matthew B. Pomaville and David D. Lent


Hiroyuki Ai, Ajayrama Kumaraswamy, Tsunehiko Kohashi, Hidetoshi Ikeno and Thomas Wachtler


Mu-Yun Wang, Lars Chittka and Thomas C. Ings


# Editorial: The Mechanisms of Insect Cognition

#### Lars Chittka<sup>1</sup> \*, Martin Giurfa<sup>2</sup> \* and Jeffrey A. Riffell <sup>3</sup> \*

*<sup>1</sup> School of Biological and Chemical Sciences, Queen Mary University of London, London, United Kingdom, <sup>2</sup> Research Center on Animal Cognition, Center of Integrative Biology, CNRS - University Paul Sabatier - Toulouse III, Toulouse, France, <sup>3</sup> Department of Biology, University of Washington, Seattle, WA, United States*

Keywords: brain, cognition, computation, learning, memory, neuroscience

**Editorial on the Research Topic**

#### **The Mechanisms of Insect Cognition**

Insects have miniature brains, but recent discoveries have upturned historic views of what is possible with their seemingly simple nervous systems (Giurfa, 2013). Phenomena like tool use (Loukola et al., 2017; Mhatre and Robert), face recognition (Chittka and Dyer, 2012; Avarguès-Weber et al.), numerical competence (Skorupski et al., 2018; Howard et al., 2019), and learning by observation (Leadbeater and Dawson, 2017) beg the obvious question of how such feats can be implemented in the exquisitely miniaturized bio-computers that are insect brains. Bringing together researchers from insect learning psychology and neuroscience in a common forum in this Research Topic was envisaged to inject momentum into this dynamic and growing field. In selecting our authors, we have deliberately chosen not to focus on seniority and pontification—we have made a concerted effort to recruit junior authors, as well as scientists from diverse countries (of 17 nations) including some where current governments have erected substantial obstacles to basic research.

Many remarkable behavioral feats of insects concern their navigation. Examples include insects that can memorize multiple locations that are many kilometers apart (Collett et al., 2013), and some can find and remember efficient routes to link multiple destinations (Lihoreau et al., 2012). Understanding the neural underpinnings of such spatial memory feats is a fascinating endeavor, and this field forms thus one of the key areas of our Research Topic.

Some of the champions of insect navigation are ants which can find their way over long distance in either cluttered environments, or, as in some desert species, in featureless terrain (Freas and Schultheiss). Nocturnal bull ants can even make adaptive use of the third dimension—when they are viewing the familiar scenery from above, while climbing on trees (Freas et al.). Using modeling, Le Moël et al. demonstrate that a tiny area of the brain of ants and other insects, the central complex, can integrate landmark and vector memories as well as a sun compass (which requires knowing the time of day). Because of its role in integrating external input with internal simulation of the environment, the central complex has recently been pinpointed as a possible seat of consciousness in insects (Barron and Klein, 2016). An argument for this notion comes from the work of Libersat et al. on parasitoid wasps that highjack their insect hosts' brains to disable normal behavior, manipulating them in such a way as to benefit the parasites. The wasps sting their hosts into a brain area around the central complex, which aborts all self-initiated behavior in the hosts.

But Hymenoptera (bees, ants, and wasps) are not the only insects that are excellent navigators. Pomaville and Lent, working with paradigms such as split-brain cockroaches, discover that these insect have multiple navigation systems in their brains, which are integrated into a coherent whole. The kissing bug displays considerable versatility in spatial avoidance learning (Minoli et al.). While in some cases, innate responses to certain stimuli are unaffected by experience, the bugs can learn

#### Edited and reviewed by:

*Sarah Till Boysen, The Ohio State University, United States*

#### \*Correspondence:

*Lars Chittka l.chittka@qmul.ac.uk Martin Giurfa martin.giurfa@univ-tlse3.fr Jeffrey A. Riffell jriffell@uw.edu*

#### Specialty section:

*This article was submitted to Comparative Psychology, a section of the journal Frontiers in Psychology*

Received: *11 November 2019* Accepted: *21 November 2019* Published: *05 December 2019*

#### Citation:

*Chittka L, Giurfa M and Riffell JA (2019) Editorial: The Mechanisms of Insect Cognition. Front. Psychol. 10:2751. doi: 10.3389/fpsyg.2019.02751*

**5**

to associate stimuli in a wide variety of sensory modalities as predictors of aversive stimuli. Indeed, using Drosophila as a model, Gorostiza shows just how outdated the idea of insects as reflex machines really is: almost all behavior routines previously thought to be innate, also have cognitive components. This is even the case for some spatial behavior routines that were previously thought to be entirely governed by instinct—the construction of hexagonal comb structures in bees. From the exploration of the historic literature— Gallo and Chittka found experimental evidence, over 200 years old and buried in largely forgotten literature, showing that honeybees might display a mental planning ability in their comb constructions (Huber, 1814).

Several papers of the volume show how complex cognitive functions can be implemented with clever behavioral shortcuts, using minimal neural circuitry: e.g., the Mhatre and Robert shows the tricks by which insects can manufacture tools, Avarguès-Weber et al. discover the psychological strategies by which some insects can recognize faces, and Guiraud et al. determine how bees can solve a visual concept learning tasks successfully but in a manner totally alien to humans. Insects also display "personality"—as evidenced when each individual's emphasis on speed and accuracy when choosing between foraging options under predation threat (Wang et al.), and (Tomasiunaite et al.) demonstrate how these often complex and individual behaviors can be quantified using automated computer vision methods.

A recurrent theme of the volume is that complexity can arise even from "simple" associative learning, similarly to mammals. When Drosophila larvae learn to discriminate rewarded form non-rewarded stimuli, they not only learn to search for the rewarded stimulus, but also to avoid the non-rewarded stimulus, revealing the existence of multiple simultaneous memory traces arising from the same learning (Schleyer et al.). The Prediction Error Theory (Rescorla and Wagner, 1972) originally proposed for mammals and stating that the discrepancy between the actual and the predicted reward is a main determinant of learning, also applies to cricket learning (Mizunami et al.). Similarities also occur at the molecular level as the signaling pathways leading to the formation of long-term memory in crickets share attributes with those of mammals (Matsumoto et al.).

Several contributions focus on odor learning, for example in a South American bumblebee species (Palottini et al.). Drosophila larvae, after being trained to discriminate two odors, reveal the principles of odor mixture processing based on their responses to combinations of the two odors trained (Chen et al.). Learning of multisensory stimuli is studied by Mansur et al., who adopted so-called patterning protocols, which explore insects' capacity to treat stimulus compounds as being different from the simple sum of their components.

The importance of nutritional state on learning has received considerable attention in mammals. In honeybee olfactory learning, best performance is achieved by a diet rich in essential fatty acids, as long as the omega-6:3 ratio is not high (Arien et al.). In bees, lateralization of odor processing occurs with respect to the two antennae (Frasnelli et al., 2012). Baracchi et al. extend this analysis to the processing of the sucrose reward and find a right hemisphere dominance, adding a further dimension to brain lateralization in bees.

The specific brain regions, neural circuits and genes allowing behavioral complexity in insects are explored in several contributions. The role of inhibitory signaling via GABAergic neurons in various brain regions of moths, crickets and bees is highlighted by Ai et al. The mushroom bodies are the main neural substrate for memory storage and retrieval. Focusing on the learning of a specific time of the day as predictor of food reward, Shah et al. show that the expression of the immediate early gene Egr-1 of bee mushroom bodies is regulated by the circadian clock. Suenami et al. show that types of mushroom body cells can be distinguished based on protein and genespecific differences.

Insects are able to produce highly sophisticated behavior—as highlighted by our issue—using fewer neurons than vertebrate brains. The possibility of identifying specific neural architectures as the basis for some forms of cognitive processing has a unique potential for the development of research on artificial intelligence and robotics, and possibly for the understanding of the human brain. Psychologists and neuroscientists, as well as engineers and computer scientists, are thus beginning to appreciate insects as a model system for examining how intelligence can be achieved with miniature nervous systems. The exploration of the brain mechanisms mediating complex cognition in insects is key to understanding the very nature of intelligence—in all animals, not just insects. Finally, the insect apocalypse (mass extinctions as a result of habitat destruction, pesticide use, and climate change) is global news. Insects have not just important ecosystem functions but are vital to human food security—e.g., one third of the food we consume (most fruits and vegetables) are directly contingent upon insect pollination. Understanding their cognition and the richness of their perceptual worlds adds a new slant to their conservation and research ethics.

# AUTHOR CONTRIBUTIONS

All authors listed have made a substantial, direct and intellectual contribution to the work, and approved it for publication.

# ACKNOWLEDGMENTS

Several articles of the Research Topic were submitted by its Editors. According to the journal's policies, to avoid conflicts of interest, these articles were handled by external editors Giorgio Vallortigara, Patrizia d'Ettorre, Thomas Bugnyar, and Ken Cheng. We wish to thank them for their work. We also thank the Human Frontiers Science Program (HFSP) for a joint grant (RGP0022/2014) to explore miniature brains. Collaborating on this project inspired the editors to produce this Research Topic.

# REFERENCES


**Conflict of Interest:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2019 Chittka, Giurfa and Riffell. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Behavioral Evidence for Enhanced Processing of the Minor Component of Binary Odor Mixtures in Larval Drosophila

#### Yi-chun Chen1, 2, Dushyant Mishra2†, Sebastian Gläß1† and Bertram Gerber 1, 2, 3, 4 \*

<sup>1</sup> Abteilung Genetik von Lernen und Gedächtnis, Leibniz Institut für Neurobiologie, Magdeburg, Germany, <sup>2</sup> Genetik und Neurobiologie, Biozentrum, Universität Würzburg, Würzburg, Germany, <sup>3</sup> Center for Behavioral and Brain Sciences, Magdeburg, Germany, <sup>4</sup> Institut für Biologie, Universität Magdeburg, Magdeburg, Germany

#### Edited by:

Martin Giurfa, UMR5169 Centre de Recherches sur la Cognition Animale (CRCA), France

#### Reviewed by:

Dennis Mathew, University of Nevada, Reno, United States Silke Sachse, Max Planck Society (MPG), Germany

#### \*Correspondence: Bertram Gerber

bertram.gerber@lin-magdeburg.de

#### † Present Address:

Dushyant Mishra, Department of Molecular and Cellular Medicine, Texas A&M University Health Science Center, College Station, TX, United States Sebastian Gläß, ADASTRA GmbH, Frankfurt, Germany

#### Specialty section:

This article was submitted to Comparative Psychology, a section of the journal Frontiers in Psychology

Received: 31 August 2017 Accepted: 17 October 2017 Published: 06 November 2017

#### Citation:

Chen Y-c, Mishra D, Gläß S and Gerber B (2017) Behavioral Evidence for Enhanced Processing of the Minor Component of Binary Odor Mixtures in Larval Drosophila. Front. Psychol. 8:1923. doi: 10.3389/fpsyg.2017.01923 A fundamental problem in deciding between mutually exclusive options is that the decision needs to be categorical although the properties of the options often differ but in grade. We developed an experimental handle to study this aspect of behavior organization. Larval Drosophila were trained such that in one set of animals odor A was rewarded, but odor B was not (A+/B), whereas a second set of animals was trained reciprocally (A/B+). We then measured the preference of the larvae either for A, or for B, or for "morphed" mixtures of A and B, that is for mixtures differing in the ratio of the two components. As expected, the larvae showed higher preference when only the previously rewarded odor was presented than when only the previously unrewarded odor was presented. For mixtures of A and B that differed in the ratio of the two components, the major component dominated preference behavior—but it dominated less than expected from a linear relationship between mixture ratio and preference behavior. This suggests that a minor component can have an enhanced impact in a mixture, relative to such a linear expectation. The current paradigm may prove useful in understanding how nervous systems generate discrete outputs in the face of inputs that differ only gradually.

Keywords: learning, memory, perception, compound conditioning, decision-making

# INTRODUCTION

Brains organize the integration of behavioral options, internal state including memory, and sensory information. One important boundary condition for this integration is that behavioral options are often mutually exclusive (fight or flight; approach or avoidance; going left or right), although internal states and sensory inputs can vary continuously. Here we provide an experimental handle on this process of generating discrete output in the face of inputs varying in grade, in larval Drosophila. We develop an olfactory "morphing" experiment (e.g., Steullet and Derby, 1997; Niessing and Friedrich, 2010) based on an established associative odor-sugar learning paradigm (Scherer et al., 2003; Neuser et al., 2005; review: Diegelmann et al., 2013). In that paradigm, larvae are either trained such that odor A is rewarded and odor B is not (A+/B), or they are trained reciprocally (A/B+). Typically, the larvae are then tested for their choice between the two odors. In this study, however, the larvae from both experimental groups are tested either for their preference for A in the absence of B, or for their preference for B in the absence of A, or for their preference for a mixture of A and B. The "morphing" of A into B is implemented by altering the ratio between A and B in the mixture. This provides a behavioral read-out for which of these mixtures the larvae regard as A or as B. Following earlier approaches (Mishra et al., 2010; Chen et al., 2011; Eschbach et al., 2011; Niewalda et al., 2011; Chen and Gerber, 2014), it is a distinguishing feature of our study that we choose dilutions of A and B on the basis of equal taskrelevant behavioral potency (i.e., equal learnability), rather than on the basis of procedural, physical or physiological criteria (equal dilution, equal concentration, or equal spike rate at a given stage of the olfactory pathway).

#### RESULTS

We report two series of learning experiments with a total of 38 experimental groups and a total sample size of N > 700 (each N reflecting the behavior of n = 30 larvae).

We trained the larvae by differentially rewarding benzaldehyde (BA) or hexylacetate (HA), and tested them for their preference either for BA, or for HA, or for mixtures of BA and HA at the indicated ratios (**Figure 1A**). At the chosen unit-dilutions, these two odors are equally learnable (**Figure S1A**). We first wanted to see whether for a particular BA: HA mixture ratio the larvae would regard that mixture as BA or as HA. According to the convention introduced in the Methods section, positive 1Preference scores indicate that the larvae regard the mixture as BA, whereas negative 1Preference scores indicate recognition of the mixture as HA. Results are apparently symmetrical (**Figure 1B**) in that larvae regard the mixture as BA as long as the BA: HA ratio is high, and regard the mixture as HA if the BA: HA ratio is low, while for ratios around 5: 5, 1Preference scores are close to zero. The critical question, however, is whether the larvae regard the mixture as the major component or as the minor component—irrespective of the chemical identity of the odors. To this end, we re-present the data from **Figure 1B** by "folding" the display first along its horizontal and then along its vertical midline (to facilitate comparisons with the second odor pair used in this study, the data were further normalized to the highest median thus obtained; see section Behavioral Paradigm and Presentation Of Mixtures). The resulting norm-1PREF scores differ significantly between groups (**Figure 1C**; KW-test P < 0.05, H = 51.11, df = 5) and reveal that replacing less than half of the mixture can abolish recognition of the mixture as the major component (**Figure 1C**; W-tests of the four left-most plots P < 0.05/6; for the two right-most plots P > 0.05/6).

Given that we find qualitatively the same results for 1 octen-3-ol (1-OCT-3-OL) and 3-octanol (3-OCT) as the second tested odor pair (**Figure 2**, **Figure S1B**), in **Figure 3A** we jointly present the medians of the norm-1PREF scores plotted against the proportion of the major component in the mixture. For comparison, the red stippled line shows the scores to be expected if the mixture was treated as a linear sum of its components (Y = 2X + [−1]). Defined relative to this linear expectation, an analysis across the complete dataset reveals an enhanced behavioral impact of the minor component in the mixture (**Figure 3A**; Wtest: P < 0.05). For example, for a mixture with a 0.8 proportion of the major component, the linear expectation is that the larvae should show 60% of the full score. As shown in **Figure 3B**, the scores are less than this linear expectation (**Figure 3B**; W-test: P < 0.05). Thus, our results demonstrate that a minor component can have a more-than-linear effect in a mixture.

### DISCUSSION

We found, as expected, that after differential training the larvae show higher preference for the previously rewarded than for the previously unrewarded odor. However, when the test is performed for mixtures of both odors, this difference in associative preference in favor of the respectively major component becomes less (**Figures 1C**, **2C**, **3A**). For a mixture ratio of 8: 2 (or 2: 8), recognition of the mixture as the respectively major component was largely degraded, although based on a linear account (stippled line in **Figures 3A,B**), 60% of the training effect should remain detectable. This suggests that, for the used odor pairs, the larvae treat the test mixture in a non-linear fashion, in a way that is skewed toward the minor component. That is, although the major component does dominate preference behavior toward the mixture, it does so less than linearly expected.

We have previously shown (Mishra et al., 2013) that when an odor concentration was decreased ten-fold between training and test (in the terminology of the current paper from 10: 0 to 1: 0), as much as half of the training effect remained. A similar result was found after odor-shock learning in adult Drosophila (Yarali et al., 2009). Thus, the decrement in the morphing function is unlikely to be explained solely by the comparably slight decrease in absolute concentration of the major component of the mixture.

In psychological terms, there are two extreme views on the current results which are equally compatible with the present data. Firstly, a mixture may be perceived by its elements such that in the case of our experiments both memories are addressed during mixture testing but because they are opposite in "value" they cancel each other out. Alternatively, a mixture may be perceived as a novel, unique configuration such that in our experiments the memories for the elements are not even addressed during mixture testing (for discussion, see Pearce, 1994; Redhead and Pearce, 1995; Melchers et al., 2008). Both elemental and configural modes of processing yield ecologically valid information, pertaining respectively to the presence of odor molecules and the jointness of their presence. As both these kinds of information can be or can become useful, animals and humans fittingly appear capable of both kinds of processing and of adopting them in a task-dependent manner (Livermore et al., 1997; Steullet and Derby, 1997; Gerber and Ullrich, 1999; Müller et al., 2000; Deisig et al., 2003; Giurfa et al., 2003; Tabor et al., 2004; Su et al., 2011; Münch et al., 2013; Schubert et al., 2015). We note that our behavioral results from both adult (Eschbach et al., 2011) and larval Drosophila (Chen and Gerber, 2014) do not suggest particularly strong configural effects; levels of generalization for a mixture are typically equal for both elements, and conversely an element is typically equally similar to all mixtures containing it. A more direct argument

(Continued)

#### FIGURE 1 | Continued

Preference scores of (for example) group 2 from the Preference scores of group 1, etc., for each pair of data points (displayed in B). (B) The 1Preference scores quantify associative recognition. Taking groups 1 and 2 as an example, the associative preference for BA should be higher after BA was rewarded than when it was unrewarded (positive 1Preference scores). Likewise, in groups 21 and 22, the associative preference for HA should be lower after HA was unrewarded than when HA was rewarded (negative 1Preference scores). In other words, positive 1Preference scores indicate recognition of the mixture as BA, whereas negative 1Preference scores indicate recognition of the mixture as HA. (C) Re-presentation of the data from (B) as norm-1PREF scores (for details see Materials and Methods section), indicating whether, irrespective of chemical identity, the larvae regard the mixture as the major or as the minor component. Data differ across groups (KW-test, P < 0.05, H = 51.11, df = 5); asterisks above the box plots refer to significant differences from zero in W-tests (P < 0.05/6).

is that adult Drosophila are apparently unable to solve either negative patterning discrimination tasks (both A-alone and Balone are reinforced, but AB is not: A+, B+, AB) or biconditional discrimination tasks (both AB and CD are reinforced, but AC and BD are not: AB+, CD+, AC, BD), although mixtureunique processing would enable these faculties (Young et al., 2011; Wessnitzer et al., 2012). In the absence of evidence to the contrary from (for example) summation experiments, it thus seems plausible that, without significant prior exposure to the mixture and for the tested odor stimuli and paradigm at least, Drosophila larvae perceive a binary mixture largely by its elements. We therefore suggest that during testing the opposing values of the memories of the mixture elements cancel one another out. In particular, a minor component is apparently capable of countering the impact of a quantitatively dominant component (**Figures 3A,B**). Using the present paradigm, it can now be tested whether this comes about at the level of the olfactory sensory neurons (Münch et al., 2013), within the antennal lobe (Silbering and Galizia, 2007; Fernandez et al., 2009; Olsen et al., 2010), the mushroom bodies (Honegger et al., 2011), and/ or at several of these stages (Barth et al., 2014; Schubert et al., 2015). Studied at the level of individual animals, this may provide a study case of how a simple nervous system transforms gradually differing sensory inputs into categorically different behavioral outputs (for such a study in the auditory system of rodents: Ohl et al., 2001).

#### MATERIALS AND METHODS

#### Larvae

Third instar feeding-stage Drosophila melanogaster larvae (5 days after egg laying) of the Canton Special wild-type strain were used, kept in mass culture under a 14: 10 h light: dark cycle at 25◦C and 60–70% relative humidity.

#### Petri Dishes

One day prior to the experiment, Petri dishes of 85 mm inner diameter (Sarstedt, Nümbrecht, Germany) were filled either with a solution of 1% agarose (electrophoresis grade; Roth, Karlsruhe, Germany) or with 1% agarose with 2 mol/l fructose added (Roth, Karlsruhe, Germany). Once the agarose had solidified, the dishes were covered with their lids and left at room temperature until the following day.

#### Odors and Their Unit-Dilutions

As odors, we used benzaldehyde (BA, CAS: 100-52-7), hexylacetate (HA, CAS: 142-92-7) (both from Sigma-Aldrich, Steinheim, Germany), 1-octen-3-ol (1-OCT-3-OL, CAS: 3391- 86-4) and 3-octanol (3-OCT, CAS: 589-98-0) (both from Merck, Hohenbrunn, Germany, purity 99%). The odors were diluted in paraffin oil (Merck, Darmstadt, Germany, CAS: 8012-95-1) at ratios of 1: 100, 1: 100, 1: 10,000 and 1: 100,000 respectively for BA, HA, 1-OCT-3-OL, and 3-OCT. These dilutions were chosen because earlier experiments (Mishra et al., 2013) had revealed that at these dilutions the odors support equal levels of learning. It is important to note that these dilutions, for the purpose of the rest of this paper, were defined as the baseline condition for each odor and were assigned the unit-dilution of "1." For the preparation of mixtures based on these unit-diluted odors see the section Behavioral Paradigm and Presentation of Mixtures.

On the day of the experiment, 10 µl of odor-solution was placed into custom-made Teflon containers with an inner diameter of 5 mm, and a perforated cap with 7 holes of 0.5 mm diameter, each. Containers without any odor added were denoted as empty (EM) (paraffin is without behavioral effect in our paradigm: Saumweber et al., 2011). Before the experiments started, we exchanged the regular lids of the Petri dishes with lids perforated in the center by fifteen 1 mm holes to improve aeration.

#### Behavioral Paradigm and Presentation of Mixtures

A spoon-full of medium containing larvae was put into an empty Petri dish and a cohort of 30 larvae was collected and briefly washed in distilled water. In principle (sketch above **Figure 1A**), the larvae were trained such that one odor was rewarded, and another odor was not (e.g., A+/B). Then, the larvae were tested for their preference either for A, or for B, or for a mixture of A and B. The key variable across this study was that by means of altering the relative proportions of A and B in the mixture we could see which of these mixtures the larvae regard as A, and which they regard as B.

The behavioral experiments were performed under a fume hood at 21–26◦C, under the light from a standard fluorescent lamp. The larvae were trained and tested in cohorts of n = 30 individuals for each data point, using either of two reciprocal training regimens. Taking the N = 40 cohorts of experimental group 1 of **Figure 1A** as an example, at the beginning of training we placed two odor containers filled with BA at opposite sides of a Petri dish containing agarose with fructose added (+). The larvae were placed in the middle and left free to move on the Petri dish for 5 min. They were then removed to another dish featuring containers filled with HA and with an agarose-only

P < 0.05/4 in W-tests for the norm-1PREF scores.

red stippled line indicate an enhanced impact of the major component, whereas scores below the red stippled line indicate enhancement of the impact of the minor component, relative to such a linear expectation. A test across the complete dataset represented here by the medians reveals that scores are consistently smaller than this expectation (W-test, P < 0.05, N = 312) (for the 1.0 case the median norm-1PREF score equals 1 by definition, such that they cannot be included in this analysis). Please note that, because for the 0.5 case we used an arbitrary convention as to whether the norm-1PREF scores were positive or negative (see Materials and Methods section), the respective points of the functions had to be omitted from this plot. (B) Pooled norm-1PREF scores for both odor pairs statistically tested against the linear expectation (i.e., norm-1PREF = 0.6, red stippled line) for a mixture with a 0.8 proportion of the major component. \*P < 0.05 in a W-test, N = 116.

substrate, where they also spent 5 min. This cycle of BA+/HA training was repeated two more times, using fresh Petri dishes in each case. At the end of this training, the larvae were placed in the middle of a Petri dish filled with only agarose. Odor containers were placed on opposite sides: on one side, the odor container was filled with BA, while the container was empty on the other side (BA–EM) (the sidedness of the placement of these containers was balanced across repetitions of the experiment). After 3 min, the larvae on each half of the dish were counted to calculate a Preference score as:

#### (i) Preference BA = (#BA − #EM)/#Total

In this formula, # designates the number of larvae on the corresponding side of the dish. Preference BA values thus range from −1 to 1; positive values indicate approach to BA, negative ones indicate avoidance.

Alternately, we trained larvae reciprocally (group 2 in **Figure 1A**: BA/HA+) (the sequence of training trials was balanced across repetitions of the experiment; that is, in half of the cases training was as in the example above, whereas in the other half of the cases it was HA/BA+ and HA+/BA, respectively). Thus, the associative recognition of BA would be revealed by group 1, which was rewarded upon presentation of BA, having a stronger preference for BA than the reciprocally trained group 2, which had received presentations of BA without the reward. This difference in Preference BA scores between the reciprocally trained groups was quantified as:

$$\text{(ii)} \quad \Delta \text{Preferences} = (\text{Preference}\_{\text{BA}, \text{group1}} - \text{Preference}\_{\text{BA}, \text{group2}}) / 2$$

Thus, the associative recognition of BA is shown by positive 1Preference scores. This reciprocal training procedure, comprising both group 1 and group 2, is designated henceforth in an abbreviated convention as:

#### TRAINING BA/HA TEST BA − EM

The same procedure was used for all those groups for which BA featured as the major component of the mixture in the test (groups 3–10).

For those groups for which HA was the major component (groups 13–22), the experiments were correspondingly performed as:

#### TRAINING BA/HA TEST HA − EM

The above equations were modified accordingly, for example for groups 21 and 22.

(iii) Preference HA = (#HA − #EM)/#Total

(iv) 1Preference = (Preference HA, group21 − Preference HA, group22)/2

Thus, the associative recognition of the mixture as HA is shown by negative 1Preference scores: for example, group 21 received unrewarded presentations of HA and should therefore show lower Preference HA scores than the reciprocally trained group 22, which received rewarded presentations of HA, leading to a negative 1Preference score.

In groups 11 and 12, testing was carried out with a 5: 5 mixture of BA and HA. In these cases, we opted to use formulae (i and ii).

To quantify whether, irrespective of the chemical identity of the mixture constituents, the larvae regard the mixture as the major component or as the minor component, we multiplied the 1Preference scores of groups 13-22 by (−1), and termed these scores 1PREF (for groups 1–12, 1Preference = 1PREF). In other words, the display in **Figure 1B** was "folded along its horizontal midline." These 1PREF scores could then be combined for the corresponding mixture ratios. That is, data from groups (1,2) were combined with groups (21,22), groups (3,4) were combined with (19,20) etc., effectively 'folding the display along its vertical midline'. To allow these 1PREF scores to be compared with those obtained for another odor pair (see below), these scores were normalized to the highest median 1PREF score thus obtained (norm-1PREF). Thus, recognition of the mixture as the major component is shown by positive norm-1PREF scores, whereas negative norm-1PREF scores would imply that the larvae regard the mixture as the minor component.

Please note that comparing the 1PREF scores derived from groups (1,2) with those of groups (21,22) allows us to compare the amount of associative learning about BA with the amount of associative learning about HA, and thus to confirm that these two odors, at the chosen dilutions, are indeed equally learnable in our paradigm (**Figure S1A**).

In all cases, we kept the total volume of unit-diluted odor as 10 µl. That is, taking groups 7 and 8 in **Figure 1A** as an example, we used 8 µl of unit-diluted BA and 2 µl of unitdiluted HA. Specifically, 8 µl of unit-diluted BA was mixed with 2 µl of paraffin and added to one odor container, while to another odor container we added 2 µl of unit-diluted HA and 8 µl of paraffin. For testing, we placed these two odor containers adjacent to each other, and opposite to a single empty container.

We repeated the above experiments, making the due adjustments indicated, using the odor pair 1-OCT-3-OL and 3-OCT.

### Statistics

Data were obtained in parallel for all the groups to be compared statistically, using non-parametric analyses throughout. To test the scores against expected values we used Wilcoxon signed-rank tests (W-tests). To test for differences across multiple groups, Kruskal-Wallis tests were used (KW-tests); for pair-wise differences we used Mann-Whitney U-tests (MWU-tests). As applicable, the significance level of 0.05 was corrected to account for multiple comparisons such that an experiment-wide error rate of 5% was maintained by Bonferroni corrections. For instance, when the data of four groups were individually compared to zero, the corrected significance level was 0.05/4.

All statistical analyses were performed with Statistica 12.0 (StatSoft, Tulsa, OK, USA) on a PC. The data are visualized as box plots with the median as bold line, box boundaries as the 25/75% quantiles and whiskers as the 10/90% quantiles.

Experiments comply with applicable law of the State of Sachsen-Anhalt and the Federal Republic of Germany, and the rules of conduct of the German Science Foundation (DFG) and the Leibniz Association (WGL).

Experimenters were blinded with respect to whether the training Petri dishes contained the fructose reward or not.

Sample sizes were chosen to be about twice as high as in previous studies investigating generalization decrements after olfactory learning in larval Drosophila (Mishra et al., 2010; Chen et al., 2011; Chen and Gerber, 2014) because we expected more moderate effects from changing mixture ratios than from changing the identity of the olfactory stimulus altogether.

## DATA AVAILABILITY

The data underlying the presented figures and used for statistical analyses are available from **Table S1**.

# AUTHOR CONTRIBUTIONS

Conceived of study: YC, DM, BG. Performed experiments: YC and DM. Analyzed data: YC and SG. Prepared figures: YC and SG. Wrote manuscript: YC, DM, SG, and BG.

# FUNDING

Institutional support: Leibniz Institut für Neurobiologie (LIN) Magdeburg, Wissenschaftsgemeinschaft Gottfried Wilhelm Leibniz (WGL), Universität Magdeburg, Center for Behavioral and Brain Sciences (CBBS) Magdeburg, Universität Würzburg. Project funding: Deutsche Forschungsgemeinschaft (DFG) (CRC 779 Motivated Behavior, GE1091/4-1), Bundesministerium für Bildung und Forschung (BMBF) (Bernstein Focus Insect Inspired Robotics), European Commission (FP7-ICT project Miniature Insect Model for Active Learning [MINIMAL]) (to BG), STIBET program of the Deutscher Akademischer Austauschdienst (DAAD, via Graduate School Life Sciences, Universität Würzburg) (to YC). The publication of this article was funded by the Open Access Fund of the WGL.

# ACKNOWLEDGMENTS

We thank K. Gerber and V. M. Saxon for tireless help with the behavioral experiments, D. Galili (Cambridge), M. Schleyer and A. Yarali (LIN) for comments, and R.D.V. Glasgow (Zaragoza, Spain) for language editing.

#### SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpsyg. 2017.01923/full#supplementary-material

Figure S1 | (A) The 1Preference scores for the 10: 0 and 0: 10 "mixture" ratios allow the learnability of BA to be compared with the learnability of HA. To this end, the 1Preference scores for HA (the right-most plot in Figure 1B) were multiplied by −1; for BA (the left-most plot in Figure 1B), 1Preference = 1Preference. ns: MWU-test: P > 0.05 U = 679, N = 40, 40. (B) Same as in (A) for 1-OCT-3-OL and 3-OCT. ns: MWU-test: P > 0.05, U = 174, N = 19, 19.

Table S1 | The table presents the data underlying the displayed figures and reported statistics.

#### REFERENCES


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2017 Chen, Mishra, Gläß and Gerber. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# The View from the Trees: Nocturnal Bull Ants, Myrmecia midas, Use the Surrounding Panorama While Descending from Trees

#### Cody A. Freas<sup>1</sup> \*, Antione Wystrach<sup>2</sup> , Ajay Narendra<sup>1</sup> and Ken Cheng<sup>1</sup>

<sup>1</sup> Department of Biological Sciences, Macquarie University, Sydney, NSW, Australia, <sup>2</sup> Research Centre on Animal Cognition, Centre for Integrative Biology, CNRS, University of Toulouse, Toulouse, France

Solitary foraging ants commonly use visual cues from their environment for navigation. Foragers are known to store visual scenes from the surrounding panorama for later guidance to known resources and to return successfully back to the nest. Several ant species travel not only on the ground, but also climb trees to locate resources. The navigational information that guides animals back home during their descent, while their body is perpendicular to the ground, is largely unknown. Here, we investigate in a nocturnal ant, Myrmecia midas, whether foragers travelling down a tree use visual information to return home. These ants establish nests at the base of a tree on which they forage and in addition, they also forage on nearby trees. We collected foragers and placed them on the trunk of the nest tree or a foraging tree in multiple compass directions. Regardless of the displacement location, upon release ants immediately moved to the side of the trunk facing the nest during their descent. When ants were released on non-foraging trees near the nest, displaced foragers again travelled around the tree to the side facing the nest. All the displaced foragers reached the correct side of the tree well before reaching the ground. However, when the terrestrial cues around the tree were blocked, foragers were unable to orient correctly, suggesting that the surrounding panorama is critical to successful orientation on the tree. Through analysis of panoramic pictures, we show that views acquired at the base of the foraging tree nest can provide reliable nest-ward orientation up to 1.75 m above the ground. We discuss, how animals descending from trees compare their current scene to a memorised scene and report on the similarities in visually guided behaviour while navigating on the ground and descending from trees.

#### Keywords: navigation, ants, nocturnal, landmarks, foraging, scanning

# INTRODUCTION

Solitary ant foragers moving on the ground are adept at navigating through their environment, both while searching for resources and when returning to their nest. Ants that forage alone show the ability to utilise multiple visual navigational systems to reach desired locations. These mechanisms include path integration using the celestial compass (Collett and Collett, 2000; Wehner and Srinivasan, 2003), systematic search (Wehner and Srinivasan, 1981; Müller and Wehner, 1994;

#### Edited by:

Lars Chittka, Queen Mary University of London, United Kingdom

#### Reviewed by:

Zhanna Reznikova, Institute of Systematics and Ecology of Animals (RAS), Russia Paul Graham, University of Sussex, United Kingdom

> \*Correspondence: Cody A Freas freascody@gmail.com

#### Specialty section:

This article was submitted to Comparative Psychology, a section of the journal Frontiers in Psychology

Received: 10 October 2017 Accepted: 08 January 2018 Published: 25 January 2018

#### Citation:

Freas CA, Wystrach A, Narendra A and Cheng K (2018) The View from the Trees: Nocturnal Bull Ants, Myrmecia midas, Use the Surrounding Panorama While Descending from Trees. Front. Psychol. 9:16. doi: 10.3389/fpsyg.2018.00016

Schultheiss et al., 2013) and landmark-based navigation (Wehner, 2003; Collett et al., 2006; Collett, 2012; Schultheiss et al., 2016).

Landmark based navigation has been widely studied in diurnal ants (Wehner et al., 1996; Fukushi, 2001; Wehner, 2003; Cheng et al., 2009; Collett, 2010; Bühlmann et al., 2011; Wystrach et al., 2011a,b, 2012; Lent et al., 2013; Narendra et al., 2013; Schultheiss et al., 2016; Freas and Cheng, 2017; Freas et al., 2017c), and the current knowledge of landmark use in ants that forage nocturnally is expanding (Reid et al., 2011; Warrant and Dacke, 2011; Freas et al., 2017a,b; Narendra and Ramirez-Esquivel, 2017; Narendra et al., 2017). What these studies have in common is that they explore navigational behavior that occurs chiefly in two dimensions while ants are travelling to goal locations on the ground. Yet foragers of multiple species, most notably those of the Myrmecia genus, travel vertically up onto their foraging tree to feed and then must successfully descend to return to the nest (Reid et al., 2011; Narendra et al., 2013; Freas et al., 2017a,b). Nocturnal species of this genus have the added challenge of completing this feat during the evening and morning twilight when visual cues are less salient compared to those used by diurnal species (Reid et al., 2011, 2013; Freas et al., 2017a,b; Narendra et al., 2017).

The study of visually directed behaviour while moving vertically has been little studied outside a few vertebrates (Jeffery et al., 2013; Yartsev and Ulanovsky, 2013). In ant species that forage predominantly on the ground, three-dimensional research has focused primarily on the ability of the path integrator to account for the slope of the ground surface during distance estimation (Wohlgemuth et al., 2001; Wintergerst and Ronacher, 2012). Navigating desert ants appear very adept at integrating terrain slope into their homeward vector, but have not been shown to use landmark cues when foragers are not oriented horizontally. The study of three-dimensional navigation using visual landmark cues is limited to work on the neotropical ant Cephalotes atratus L.. This species lives in nests high in the forest canopy, and workers that jump off the trunk direct their fall back to the same tree farther down. These ants have been shown to use landmark-based cues to direct their fall back to the tree trunk, yet appear to orient their bodies horizontally during the fall and may navigate only during this period (Yanoviak et al., 2005; Yanoviak and Dudley, 2006). In the red wood ant, Formica lugubris, foragers have been shown to use both chemical and terrestrial cues while ascending and descending trees, yet which terrestrial cues are in use remains unknown (Beugnon and Fourcassié, 1988; Fourcassie and Beugnon, 1988).

Here, we investigate whether foragers of the night-active Myrmecia midas actively navigate while foraging vertically on a tree face. M. midas foragers rely primarily on landmark cues when navigating to the nest while on the ground (Freas et al., 2017a), and have also been shown to use polarised skylight pattern to compute a homeward vector while on-route (Freas et al., 2017b). However, nothing is known about their behaviour while on a foraging tree. Nests of this species are located in the ground, at the base of a tree trunk. Some individuals forage directly on this 'nest-tree,' while other individuals navigate first along the ground before climbing up into a nearby tree's canopy. First, we examined whether foragers displaced on the vertical tree face position themselves toward the nest direction during their descent to the ground. Next, we tested foragers' descents when the terrestrial cues and celestial cues were in conflict. Then, we tested a subset of each nest's foragers that forage on the nesttree (Freas et al., 2017a). Next, to exclude the use of potential cues beyond the surrounding terrestrial cues, we blocked these terrestrial cues around the nest tree and recorded forager descents without access to the panorama. We also analysed pictures of the visual panorama at different heights and positions on the tree to discover whether nest-oriented views stored while foragers are on the ground contain sufficient information for nest-ward orientation while on the tree. Finally, we describe behaviours foragers exhibit while descending the tree, which appear to be similar to the scanning behaviours previously described on the ground (Wystrach et al., 2014; Zeil et al., 2014).

# MATERIALS AND METHODS

#### Field Site and Study Species

Experiments were conducted from September 2015 to October 2016 on three M. midas nests located in forested areas of the Macquarie University campus in Sydney, Australia (33◦ 460 11<sup>00</sup> S, 151◦ 060 40<sup>00</sup> E; Freas et al., 2017a,b). All three nests were located within a 200 m<sup>2</sup> area and foragers at each nest foraged on trees within a 15 m radius (typically ≤ 5 m) of the nest entrance. M. midas inhabits wooded areas consisting of Eucalyptus trees with understories clear of vegetation. All forager collections took place during the evening twilight and all testing occurred during the next morning after sunrise for adequate visibility during testing.

#### Foraging Tree Tests

To determine whether foragers travelling on the foraging tree actively navigate to position themselves toward the nest direction during their descent, we collected foragers travelling to a neighbouring foraging tree as they reached the tree base. These individuals were displaced to four sides of the tree face and their homeward paths were observed. This experiment was first conducted on 60 individuals (15 per displacement site) from Nest 1 and then the experiment was repeated on another 40 individuals (10 per displacement site) from Nest 2. During evening twilight, outbound foragers were collected just as they climbed onto their foraging tree located 3 m from the nest entrance at Nest 1 and 4 m from the nest entrance at Nest 2. Foragers were marked with a small amount of paint (TamiyaTM, Japan) to prevent retesting. Marked foragers were held overnight in a plastic phial with a small amount of sugar water in a darkened box. The next morning, beginning at 9 am AEST and ceasing at noon, foragers were displaced to one of four sites on the foraging tree face 2 m above ground level. The four displacement sites were designated on the tree face in relation to the nest location (0, 90, 180, and 270◦ ) with 0◦ being the nest direction and increasing clockwise. Foragers were released from the phial and allowed to climb out of the phial and onto the tree. Once on the vertical tree face, foragers were allowed to return to the nest by climbing down the tree to the ground. As the forager descended the tree, its path was marked at 1 m above ground level, ground level, and 20 cm away from the tree, and directional measurements were recorded at these three points using a smartphone-housed digital compass. Once the forager had travelled 20 cm from the foraging tree it was observed for the remainder of its path to ensure that all individuals returned to the nest entrance.

# Cue Conflict Tests

fpsyg-09-00016 January 23, 2018 Time: 17:16 # 3

In our second testing paradigm, we collected 30 foragers at Nest 1 in a similar procedure to the first experiment. Foragers were allowed to leave the nest and travel to their foraging tree located 4 m from the nest entrance. At the base of this foraging tree, these foragers were collected, marked and stored overnight. The next morning, foragers were displaced to the tree located just above nest location (nest tree). It was assumed that these foragers have some previous experience of the panorama at this site due to the proximity to the nest. Foragers were released onto the face of the nest tree, 2 m above ground level, in one of two displacement sites, designated in relation to the nest location (0◦ , n = 15; 180◦ , n = 15) with 0◦ being the nest direction. This testing regime was conducted on foragers with an acquired homeward vector as ants were captured 4 m from their nest and our displacements put this vector in ∼90◦ conflict with the terrestrial cues. Identical to previous tests, foragers were released from their phial and allowed to climb onto the nest tree face. Once vertical, foragers were allowed to return to the nest by climbing down the nest tree. As the forager descended the tree, its path was marked at 1 m above ground level and ground level, and directional measurements were recorded at these points. Once ants reached ground level they were observed to ensure all individuals entered the nest.

# Nest Tree Foragers/Landmark Blocking Experiment

The third experiment focused on a subset of ants (n = 20) that forage on the tree directly above the nest entrance (Nest 3). These foragers were allowed to leave the nest and travel the short distance to the nest tree (10 cm). Once the forager climbed onto the nest tree at 1.5 m, it was collected in a phial, marked on the gaster to prevent retesting and held overnight with food in an identical procedure to previous tests. The next morning, these foragers were displaced individually onto the nest tree but 180◦ from the nest direction, 1.5 m from the ground. In this condition, foragers' full paths on the tree face were recorded by placing small markers just behind the forager as they travelled around the tree face and down to the ground. These markers were placed approximately 10 cm apart along the path and stopped once the individual touched the ground. For each marker, we recorded the height and direction in relation to the nest entrance. Forager paths were calculated at every 10 cm from the release point to the ground and these positions were used for orientation analysis. After testing, foragers were observed as they returned to the nest entrance.

The landmark blocking condition was conducted on a separate group of nest tree foragers at Nest 3 (n = 22). Foragers were again allowed to travel the short distance to the nest tree (10 cm). Once the forager climbed onto the nest tree, they were collected, marked and fed, identical to the previous condition. Before testing, (4) 2 m long tent poles were anchored into a 1.5 m × 1.5 m square around the nest tree, ∼75 cm from the tree trunk. A 2 m high thick plastic screen was attached to the pole tops and then anchored to the ground using metal posts. This screen was suspended off the ground by a few centimetres to allow for ants to travel underneath. This set up blocked the surrounding terrestrial cue availability below the 2 m mark on the nest tree, yet did not block the view of the canopy above or any other cues on the nest tree itself. Additionally, nest tree foragers were selected for this condition as the nest entrance was located at the base of the tree (10 cm) and was well within the enclosed square created by the plastic sheet, allowing foragers access to any cues the nest presents. After collection, foragers were displaced on to the tree face opposite the nest site (180◦ ), and 1.5 m off the ground. Foragers' full paths were recorded using the same methods as in the unblocked condition. After testing, foragers were allowed to search for the nest and upon failure after 3 min. were collected and returned to the correct nest entrance location and allowed to enter the nest.

## Image Analysis: Information Available from the Foraging Tree

For all three nests, we quantified the mismatch in the panoramic scenes between nest-oriented views from the ground at the base of the foraging tree and at different elevations and compass directions on the trees where the ants were tested. To accomplish this, we collected a nest-oriented panoramic image at the base of the foraging tree. We then collected panoramic images at the four cardinal directions on the tree (0, 90, 180, and 270◦ ) at both 1 m and 1.75 m in height. The panoramic image measured 360 pixels width and 117 pixels height (roughly 50 pixels and 67 pixels below and above horizon, respectively) and were down sampled to a resolution of 1 pixel per degree. The images were converted to grayscale by keeping the blue colour channel only. This diminishes differences between clouds and blue sky but maintains high contrasts between terrestrial objects and the sky. Rotational image difference functions (rotIDFs) were calculated by using the sum of the absolute difference in pixel intensity between the reference and test images, for all possible rotations of the test images (in one-degree steps) using custom written scripts in MATLAB (for further details, see Zeil et al., 2003, 2014; Stürzl and Zeil, 2007).

# Scanning Behaviour

In order to describe the scan-like behaviour on the tree face, individual foragers were recorded both while on the tree face after displacement and on a vertically oriented board. Forager scans were recorded using a free held camera (PowerShot G12, CanonTM). Foragers were recorded after local off-route displacement on their foraging tree.

#### Statistical Procedure

Data from all experiments were analysed with circular statistics (Batschelet, 1981; Zar, 1998) using the statistics package Oriana Version 4 (Kovach Computing ServicesTM). Rayleigh's Tests were

conducted on foragers' positions on the tree face, testing if data met the conditions of a uniform distribution (p > 0.05). If data were not uniform, we tested whether positioning on the tree face was significantly clustered around the nest direction using V-tests, with alpha set at p = 0.05. We also examined if the predicted direction (0◦ ) fit within the 95% confidence interval of the foragers' positions during descent to further test positioning toward the nest (Watson Test). When an ant abandoned its descent to travel back up the tree (see blocking condition), only the positions of the individual's final descent were used for analysis.

RESULTS

Individuals placed on the tree face at the displacement sites initially paused for a short period. After this pause, foragers typically moved a short distance (usually up the tree 10–30 cm) away from the displacement point and then paused again and performed what we classify as scanning behaviours on the tree face (described below). Following this scanning behaviour, the forager moved along the tree face descending to the ground. During their descent, foragers typically performed at least one more scan-like behaviour before reaching the ground.

### Foraging Tree Tests

At both the 1m height and as they reached the ground at 0 m, Nest 1 foragers' positions on the tree face in the 0, 90, and 270◦ displacement conditions were non-uniform and significantly clustered to the nest's direction at 0◦ . Additionally, in these three conditions at both heights (1 and 0 m), the nest direction fell within the 95% confidence interval of the forager's positions (**Table 1** and **Figures 1A,B,D–F,H**). In the 180◦ condition, foragers' positions when crossing the 1 m height were uniform and not directed to the nest direction at 0◦ (**Table 1** and



**Figure 1C**). Yet as foragers in the 180◦ condition reached the ground, their positions on the tree were significantly nonuniform and clustered to the nest's direction at 0◦ . The nest direction also fell within the 95% confidence interval of the foragers' positions at 0 m (**Table 1** and **Figure 1G**). After reaching 20 cm from the tree base, forager paths in all four conditions at Nest 1 were grouped toward the nest entrance (**Table 1**) and all individuals immediately travelled the 3 m back to and entered the nest.

At Nest 2, foragers' positions on the tree face in all displacement conditions (0, 90, 180, and 270◦ ) were significantly non-uniform and significantly clustered to the nest's direction at 0◦ as they crossed to the 1 m height marker. Additionally, the nest direction fell within the 95% confidence interval of the foragers' positions at 1 m high in all conditions (**Table 1** and **Figures 2A–D**). Nest-ward positioning continued as foragers reached the ground, with all conditions showing significant nonuniformity and significant cluster toward the nest direction. Additionally, the nest fell within the 95% confidence interval of the foragers' positions (**Table 1** and **Figures 2E–H**). At Nest 2, once foragers had reached 20 cm from the tree, all individuals were oriented to the nest direction at 0◦ (**Table 1**), travelled in a straight path to the nest entrance and entered.

At the ground, foragers typically did not stop to scan again but continued on in their current direction. In all conditions foragers immediately returned to the nest entrance and entered the nest.

#### Cue Conflict Tests

To test if foragers position themselves toward either the terrestrial or celestial cues during their decent, we displaced foragers off their foraging route in order to put these cue sets in 90◦ conflict. Individuals foraging away from the nest and displaced on the nest tree showed significant nest directed positioning on the tree face at 1 m above ground level. Positions on the tree in both the 0 and 180◦ displacement conditions were significantly nonuniform and significantly grouped to the nest direction at 0◦ . This pattern continued as the foragers reached the ground, with foragers' positions being significantly directed to the nest location and non-uniform. In both conditions and at both the 1 m height and at ground level (0 m), the nest direction fell within the 95% confidence interval of foragers' positions on the tree (**Table 1** and **Figures 3A–D**). Foragers in both the 0 and 180◦ conditions showed no evidence of using their celestial based vector while positioning themselves on the tree (at 270◦ ). After descending the tree, all foragers found and entered the nest (15 cm from the tree). At the ground, foragers continued on in their current

direction. In all conditions foragers immediately returned to the nest entrance and entered the nest.

# Nest Tree Foragers/Landmark Blocking Experiment

Nest tree foragers displaced to the opposite side of the tree (180◦ ) from the nest tree at 1.5 m with access to the surrounding terrestrial cues behaved similarly to foragers that travel away from the nest to forage on a different tree. Foragers initially paused at the release point, and then moved a small distance, where they performed scan-like behaviours. These continued intermittently during the forager's decent. At the 1.4 m height, after a 10 cm decent, foragers showed uniform positioning around the tree and were not oriented to the nest site (**Table 1** and **Figures 4A**, **5A**). This uniform distribution continued at the 1.3 m, and 1.2 m heights (Rayleigh test, P > 0.05; V-test, P > 0.05). At 1.1 m, forager positions were still uniform (Rayleigh test, Z = 1.754, P = 0.174) but were significantly clustered to the nest direction, and the nest location was within the 95% confidence interval of forager positions (V-test, V = 0.295, P = 0.031). At the 1 m height, forager positions on the tree face became significantly non-uniform and significantly grouped around the nest direction at 0◦ (**Table 1** and **Figures 4C**, **5A**). This non-uniform and clustered pattern persisted at all 10 cm height measurements from 1 m to ground level with foragers significantly positioned on the nest side of the tree (1 m – 0 m; Rayleigh test, P < 0.001; V-test, P < 0.001; **Table 1** and **Figures 4E**, **5A**). At all heights between the 1 m and ground level measurements, the nest direction fell within the 95% confidence interval of foragers' positions on the tree. Once foragers had completed their descent, all individuals found and entered the nest (10 cm from the tree).

When the surrounding terrestrial cues were blocked, nesttree foragers displaced to the opposite side of the tree (180◦ ) behaved differently from previous conditions. Foragers typically scanned once near the displacement point. After this, half of the foragers tested (n = 10) travelled up the trunk above the 2 m-blocked height before beginning to perform more scans. As a whole (n = 20), foragers did not orient to the correct nest direction at any height 1.4–0 m during their descent (1.4, 1, and 0 m; **Table 1** and **Figures 4B,D,F**, **5B**). At all heights, forager positions on the tree met conditions of a uniform distribution (1.4 – 0 m, Rayleigh test, P > 0.05) and were not significantly oriented in the direction of their home vector at 0◦ (1.4–0 m, V-test, P > 0.05). As foragers reached the ground, they did not travel to the nest entrance located within

the landmark-blocking arena but instead performed looping paths, some even returning back up the tree. After 3 min, two individuals found the nest entrance and the rest were collected and moved to the nest entrance where they willingly entered.

Focusing only on those foragers that responded to the blocked panorama by ascending the tree to 2 m or higher (**Figure 5B**), when foragers first descended from 2 m or higher, they were positioned toward the nest site at 190 cm (V-test, V = 0.745, P < 0.001). This nest-ward positioning continued at all heights through 1.4 m height (V-test, V = 0.578, P = 0.004) until the 1.1 m height where forager positions were no longer non-uniform (Rayleigh test, Z = 0.504, P = 0.616) and no longer clustered to the nest side of the tree (V-test, V = 0.203, P = 0.186). These foragers' positions were uniform and not clustered toward the nest at any height between 1 m (Rayleigh test, Z = 0.559, P = 0.583; V-test, V = 0.132, P = 0.282) and 0 m (Rayleigh test, Z = 0.974, P = 0.387; V-test, V = −0.177, P = 0.782). Foragers that did not ascend above the blocking screen (n = 10) were

not positioned toward the nest at any height (V-test, 1.4 m, V = −2.827, P = 0.988; 1 m, V = −1.474, P = 0.929; 0 m, V = −0.862, P = 0.802).

the forager reaches the ground with the surrounding landmarks blocked.

# Panoramic Image Analysis: Information Available from the Foraging Tree

For all three nests, when comparing the nest-oriented panoramic views from the base of the tree to nest-oriented panoramic views at 1 m and 1.75 on the tree, we found that at both heights on the tree, the rotIDFs showed a distinct valley of minimum of mismatch (i.e., best matching direction) that was directed toward the nest [**Figures 6A,B** (green and red curves)]. This shows that directional information can be recovered up to 1.75 m (at least) from a visual memory acquired at the base of the

foraging tree. We then analysed whether animals can recover nest oriented views from different compass directions around the tree (0◦ = nest). At both 1 and 1.75 m on the tree, the views available at the other directions, 90◦ (green), 180◦ (black), and 270◦ (brown), do not generate a clear minima when compared with a view at the base of the tree (**Figures 7A,B**).

## Scanning Behaviour

While ants were on the tree face, foragers exhibited several kinds of scanning behaviours, the common characteristic of which was a shift of the body and head to bring the head's orientation at or near the horizontal plane. With the head at or close to horizontal, individuals then slowly rotated their head horizontally across the field.

The first kind of scan-like behaviour exhibited by these foragers was to use a piece of the tree's structure, such as a jutting piece of bark, a knot, or burl, creating a horizontal space at the top at which individuals can orient their entire body horizontally and then slowly shift their head across the horizontal plane (**Figure 8A**). This behaviour was environment-dependent and could occur at any point during the foragers' descent.

The second kind of scan-like behaviour, dubbed downward pitch scans, occurred as the individual reached the top of a bark strip or other structure and was oriented upward. Individuals lowered the pitch of their head while the body remained vertical, allowing individuals to bring the head close to the horizontal plane (**Figure 8B**). This behaviour was also environment-dependent but typically occurred during the initial portion of the foragers' route when some foragers travelled upward from the displacement site.

The third kind of scan-like behaviour, termed head roll scans, occurred as foragers were travelling horizontally across the vertical tree face. Foragers altered their head position by rolling the head toward the tree face, bringing the tree side of their head down and positioning their head close to the horizontal plane. From here, individuals slowly moved their head across the horizontal plane to scan (**Figure 8C**). This behaviour typically occurred when foragers were not yet on the nest side of the tree.

The final kind of scan-like behaviour, labelled the push up or upward pitch scan, was observed on the vertical tree face with the individual oriented down with the head positioned below the body. The individual extended its front legs, pushing its body and head away from the tree face. The individual's head pitched upward, reaching at or near the horizontal plane. In this position, the individual would slowly move its head across the field (**Figure 8D**). The upward pitch scan was usually observed as foragers reached the side of the tree facing the nest. These behaviours would continue throughout the forager's descent when on their descending route.

# DISCUSSION

In the current study, we show that M. midas foragers successfully orient to the nest side of their foraging tree during their

FIGURE 7 | Quantifying panorama changes at the four displacement directions and at two elevations on the foraging tree at the three nests. (A) Panoramic images at the base of the foraging tree (blue), 1 m in height at 0◦ (red), 90◦ (green), 180◦ (black), 270◦ (orange) and 1.75 m in height at 0◦ (red), 90◦ (green), 180◦ (black), 270◦ (orange). Nest orientation is at the centre of each image and images were downscaled to 1 pixel per 1◦ to resemble the ant's visual acuity, filtered through only the blue colour channel and oriented with the nest centred. (B) The rotIDF compares the root mean square pixel difference between the panorama at the base of the foraging tree with itself, and the foraging tree at both 1 and 1.75 m at each direction. The nest direction in all comparisons is centred at 0◦ .

descent. Correct nest directed positioning appears to occur well before foragers reach the ground, with foragers' positions grouped toward the nest direction at the 1-m height and at ground level. This ability appears to extend beyond the forager's current foraging tree as individuals displaced from their foraging tree to the nest tree also successfully positioned themselves

toward the nest direction both at 1-m height and at ground level. Even nest-tree foragers, which show evidence of reduced navigational knowledge on the ground (Freas et al., 2017a), are able to successfully orient while on their foraging tree above the nest entrance. Visual terrestrial cues appear to be critical to this navigational ability, as when the surrounding terrestrial cues were blocked, foragers were unable to successfully orient toward the nest entrance. Analysis of the panorama at different foraging heights suggests that ants can obtain nest orientation information at both 1 and 1.75 m above the ground, provided they are on the nest-facing tree face (0◦ ). Finally, use of the surrounding terrestrial cues fits with behaviour on the tree as foragers appear to actively scan while on the tree, bringing their head orientation to or near the horizontal plane and then slowly rotating it across the field.

When M. midas foragers are displaced in a local environment on the ground, they are able to successfully use the surrounding landmark cues to orient toward the nest (Freas et al., 2017a). Our results suggest this ability extends to elevation-based displacements. The ability to orient to familiar landmarks after vertical displacement has been previously shown in the desert ant M. bagoti (Schwarz et al., 2014), a species that forages on the ground almost exclusively (Schultheiss and Nooten, 2013). It is currently unknown if foragers include travelling vertically up the nest tree in their learning walks or if on their first trip onto the foraging tree they perform a vertical form of turn back behaviour as is observed with ants on the ground (Nicholson et al., 1999; Graham and Collett, 2006; Müller and Wehner, 2010; Fleischmann et al., 2016, 2017) and has also been reported in bees (Lehrer, 1991, 1993).

Similar nest-ward positioning was present when foragers were displaced off their foraging route to the nest tree. Ant species inhabiting complex, landmark-rich environments typically rely heavily on terrestrial cues for navigation, with landmarks tending to suppress any accumulated vector information (Wehner et al., 1996; Narendra, 2007; Narendra et al., 2013; Mangan and Webb, 2012). Yet in situations where the celestial based vector and terrestrial cues conflict, some species exhibit directional compromise behaviour (Narendra, 2007; Collett, 2010; Legge

et al., 2014; Wystrach et al., 2015; Wehner et al., 2016). This compromise between cues sets has not been observed in M. midas while navigating on the ground, as terrestrial cues largely dominate in a local area (Freas et al., 2017a). Yet M. midas foragers have shown evidence of vector cue use and celestial/terrestrial directional cue compromise while on their foraging route during both the outbound and inbound journeys (Freas et al., 2017b). In the current study, foragers showed similar behaviour with no evidence of using their naturally accumulated celestial based vector for positioning and their behaviours were consistent with navigation through terrestrial cues. It is worth noting that the accumulated vector lengths in this test are relatively short (4 m), but this distance is representative of the typical vector length by observed individuals at our field site (Freas et al., 2017a) and foragers have been shown to use celestial cues at these distances (Freas et al., 2017b).

The final unblocked condition tested foragers that travel straight up the nest tree to forage. These foragers have been previously shown to be unable to successfully orient when displaced locally on the ground (Freas et al., 2017a). It is believed that these foragers are naturally restricted horizontally to the nest site and either do not actively navigate during foraging or have reduced navigational abilities similar to C. bicolor digger ants, which do not forage (Wehner and Menzel, 1969; Freas et al., 2017a). The results of our unblocked condition suggest these foragers do actively navigate while foraging in the nest tree as these individuals successfully orient to the nest side of their foraging tree after displacement and this positioning occurs well before they reach the ground.

Our landmark blocking condition also tested nest-tree foragers, allowing us to keep the nest entrance and any directional cues it provides within the blocking arena and accessible to the foragers. Foragers' inability to position themselves toward the nest direction in this setup corresponds with landmark blocking experiments on the ground where foragers cannot orient to the nest when the surrounding panorama is blocked (Freas et al., 2017a). These results would also appear to exclude any scent-based cue, or local visual cues on the tree surface that could be used on their own for directional information. Our results also suggest that this species cannot use the unblocked canopy of the tree alone for directional information, at least during the final 2 m of their decent.

The use of the surrounding panorama for direction information is also supported by forager behaviour in the blocking condition before descending the tree. Foragers that immediately descended the tree (n = 10) were not positioned toward the nest at any height as expected if foragers used the surrounding terrestrial cues to orient. Foragers (n = 10) that responded to the blocking screen by first ascending above 2 m were positioned correctly but below 1 m correct positioning ceased (1-0 m). These findings suggest that the distant terrestrial cues are critical not only for a forager's initial positioning but are also involved in route maintenance during a forager's descent. It is possible that foragers must scan the surrounding visual panorama during their descent in order to maintain positioning on the tree. This would explain the scanning behaviour observed throughout forager descents in all conditions.

Our analysis of panoramic pictures revealed that sufficient visual information is available in the scene for the ants to orient on these trees. Image comparisons revealed variability across trees and locations, but overall, the information necessary to retrieve the nest direction using a terrestrial visual compass strategy (Wystrach et al., 2011b; Baddeley et al., 2012) is available. As noted earlier (Zeil et al., 2003; Schwarz et al., 2014), changes in height have little impact on the information available in these panoramic views. This stable nest-ward minimum in panorama information may also be used by bees and wasps as they ascend in height during their learning flights (Zeil, 1993a,b; Stürzl et al., 2016; Murray and Zeil, 2017). In the case of our ants, it is worth noting that using memories from the correct side of the tree is useful primarily when the ant is currently located on that side of the tree, as this position was where the best matches were obtained. It appears that rotIDF is not very powerful at predicting the nest direction when the ant is located on an unfamiliar side of a tree (90, 180, or 270◦ ), but has more predictive power when the ant is located on the familiar side (0◦ ). Even though there was no detectable minima at the 90, 180, or 270◦ positions on the tree (**Figure 7B**), ants were able to successfully guide themselves back toward their familiar corridor on the tree and then toward the nest. This reflects what is observed on the ground. Assuming that ants learn the scene when located on their habitual side of the tree, this would provide a gradient of familiarity that could be used to reach and stick stay on the nest side of the tree. Whether foragers use this gradient of familiarity (Zeil et al., 2003), the visual compass (Wystrach et al., 2011b; Baddeley et al., 2012) or other visual strategies (Wystrach et al., 2012; Horst and Möller, 2017), remains to be tested.

Scanning behaviour characterised by the rotation of the individual's head and body in place (Wystrach et al., 2014; Zeil et al., 2014) can be useful to exploit the familiarity of the surrounding visual scene. Ants perform more scans when their familiar surroundings have been altered or when the direction provided by terrestrial cues conflicts with celestial cues (Wystrach et al., 2014). In the current study, we show that this behaviour may extend beyond ground level, as individuals travelling vertically appear to actively scan while on their foraging tree. This potential behaviour, which is closely associated with the use of learnt visual cues, along with the results of the blocking condition and the panorama analysis, further indicate that the use of learnt visual cues is likely in use during forager descents. It has recently been shown that while on their foraging route members of M. pyriformis, another nocturnal Myrmecia species that relies heavily on the visual scene (Reid et al., 2011), attempt to stabilise their head horizontally while travelling en route on an uneven surface, as view similarity drops markedly as the view is rotated (Raderschall et al., 2016). This species has also been shown to perform extensive scanning behaviours during learning walks around the nest indicating scan behaviours are part of the nocturnal ant's navigational repertoire

(Narendra and Ramirez-Esquivel, 2017). Similar behaviours seem to apply to navigation on the tree in M. midas where foragers appear to attempt through multiple scanning behaviours to position their heads horizontally during scanning. These scans may serve a similar function as scans displayed on the ground (Wystrach et al., 2014; Narendra and Ramirez-Esquivel, 2017), and thus suggest that similar visual memories and strategies may be used when foraging both on ground and on trees. A future study on the foragers' ability to effectively scan while navigating along a vertical plane is warranted.

It is also important to note that the described behaviour of raising the head while vertical may also potentially involve the use of celestial cues, such as the sun's position, when they are available. Work on honeybee dancing in the Asian species Apis florea, a behaviour strongly tied to the position of the sun, has shown that when dancers are on a steep slope, these individuals rotate their head position to compensate for this slope. This compensation allows them to keep their visual field stable with the horizon while dancing (Dyer, 1985, 2002). This behaviour appears similar to what we observe in the current study, albeit without the horizontal movement of the head, which we have deemed scanning behaviour. It remains possible that foragers could also be using celestial cues as well as terrestrial cues while on the tree. M. midas foragers typically only forage in trees within 5 m of the nest and have shown no evidence of orienting to vectors of this length. In the rare case that foragers travel farther from the nest (14 m), we have only observed weak evidence of orientation to a vector (Freas et al., 2017a). As such, it may be possible that the observed scanning behaviour on the tree surface also allows foragers access to celestial cues.

Finally, the extent of these vertical navigational abilities is currently unknown, as well as at what height these individuals naturally show nest ward positioning during their descent. Observations of returning foragers in the predawn twilight suggest that foragers are oriented to the nest at heights over

#### REFERENCES


3 m, yet an analysis of this behaviour may prove difficult. M. midas nests at the field site are located in small stands of trees, interspersed with large tracks of grass. This habitat leads to large differences in skyline height surrounding the nest. These large skyline changes may not change drastically with changes in height of the viewer. Further studies into how the terrestrial cues change over larger changes in elevation are warranted.

#### CONCLUSION

The experiments in the current study show that M. midas actively and critically use the surrounding visual scene to orient and descend along the correct side of the tree. Image analysis of the visual scene on the tree shows that the scene provides sufficient information for these individuals to orient successfully using stored views. These foragers may extract this visual information during on-tree scanning behaviours where individuals scan their surroundings in the horizontal plane. Together, these findings suggest that visual navigational strategies and memory use may be similar between foragers navigating on the ground and on the tree.

# AUTHOR CONTRIBUTIONS

Experiments and analyses were designed by CF, AW, AN, and KC. CF collected all data. CF, AN, and AW analysed the data. CF, AW, AN, and KC drafted and revised the manuscript.

#### ACKNOWLEDGMENTS

This research was supported by the Australian Research Council through a Discovery grant to KC and AN (DP150101172) and a Future Fellowship to AN (FT140100221).



information during route following. J. Exp. Biol. 214, 363–370. doi: 10.1242/ jeb.049338


Yartsev, M. M., and Ulanovsky, N. (2013). Representation of three-dimensional space in the hippocampus of flying bats. Science 340, 367–372. doi: 10.1126/ science.1235338

Zar, J. H. (1998). Biostatisical Analysis, 4th Edn. Englewood Cliffs, NJ: Prentice Hall.


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2018 Freas, Wystrach, Narendra and Cheng. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Lateralization of Sucrose Responsiveness and Non-associative Learning in Honeybees

#### David Baracchi1,2 \*, Elisa Rigosi<sup>3</sup> , Gabriela de Brito Sanchez1,2 and Martin Giurfa1,2

<sup>1</sup> Centre de Recherches sur la Cognition Animale, Centre de Biologie Intégrative (CBI), Université Toulouse III Paul Sabatier, Toulouse, France, <sup>2</sup> Centre National de la Recherche Scientifique, Université Paul Sabatier, Toulouse, France, <sup>3</sup> Department of Biology, Lund University, Lund, Sweden

Lateralization is a fundamental property of the human brain that affects perceptual, motor, and cognitive processes. It is now acknowledged that left–right laterality is widespread across vertebrates and even some invertebrates such as fruit flies and bees. Honeybees, which learn to associate an odorant (the conditioned stimulus, CS) with sucrose solution (the unconditioned stimulus, US), recall this association better when trained using their right antenna than they do when using their left antenna. Correspondingly, olfactory sensilla are more abundant on the right antenna and odor encoding by projection neurons of the right antennal lobe results in better odor differentiation than those of the left one. Thus, lateralization arises from asymmetries both in the peripheral and central olfactory system, responsible for detecting the CS. Here, we focused on the US component and studied if lateralization exists in the gustatory system of Apis mellifera. We investigated whether sucrose sensitivity is lateralized both at the level of the antennae and the fore-tarsi in two independent groups of bees. Sucrose sensitivity was assessed by presenting bees with a series of increasing concentrations of sucrose solution delivered either to the left or the right antenna/tarsus and measuring the proboscis extension response to these stimuli. Bees experienced two series of stimulations, one on the left and the other on the right antenna/tarsus. We found that tarsal responsiveness was similar on both sides and that the order of testing affects sucrose responsiveness. On the contrary, antennal responsiveness to sucrose was higher on the right than on the left side, and this effect was independent of the order of antennal stimulation. Given this asymmetry, we also investigated antennal lateralization of habituation to sucrose. We found that the right antenna was more resistant to habituation, which is consistent with its higher sucrose sensitivity. Our results reveal that the gustatory system presents a peripheral lateralization that affects stimulus detection and non-associative learning. Contrary to the olfactory system, which is organized in two distinct brain hemispheres, gustatory receptor neurons converge into a single central region termed the subesophagic zone (SEZ). Whether the SEZ presents lateralized gustatory processing remains to be determined.

Keywords: Apis mellifera, behavioral lateralization, brain asymmetries, habituation, left–right asymmetries, proboscis extension response, sucrose sensitivity

#### Edited by:

Giorgio Vallortigara, University of Trento, Italy

#### Reviewed by:

Lesley J. Rogers, University of New England, Australia Elisa Frasnelli, University of Lincoln, United Kingdom

> \*Correspondence: David Baracchi david.baracchi@gmail.com

#### Specialty section:

This article was submitted to Comparative Psychology, a section of the journal Frontiers in Psychology

Received: 14 February 2018 Accepted: 14 March 2018 Published: 28 March 2018

#### Citation:

Baracchi D, Rigosi E, de Brito Sanchez G and Giurfa M (2018) Lateralization of Sucrose Responsiveness and Non-associative Learning in Honeybees. Front. Psychol. 9:425. doi: 10.3389/fpsyg.2018.00425

# INTRODUCTION

fpsyg-09-00425 March 28, 2018 Time: 12:40 # 2

Lateralization, once considered a hallmark of humans (Corballis, 1989), is a rather widespread animal phenomenon [recently reviewed in Rogers et al. (2013) and Rogers and Vallortigara (2017)]. Sensory and motor asymmetries in behavior, as well as asymmetries in the nervous system, occur in many taxa, independently of brain size or complexity (Frasnelli et al., 2012; Rogers et al., 2013; Frasnelli, 2017). It has been suggested that left–right asymmetries avoid duplicate processing of information, optimizing the computation of the nervous system and reducing the possibility of conflicting information from bilateral sensory organs (Vallortigara and Rogers, 2005). Lateralization might also be advantageous at the periphery of sensory systems, as shown by the example of nematodes where functional asymmetries of chemosensory neurons optimize chemotaxis (Suzuki et al., 2008) and are required for odor discrimination (Wes and Bargmann, 2001).

In the last decade, many studies reported the occurrence of sensory asymmetries in various invertebrate species (reviewed in Frasnelli et al., 2012; Frasnelli, 2017). Among insects, the honeybee has received particular attention in the study of asymmetries (reviewed in Frasnelli et al., 2014). Honeybee workers trained to associate either a visual or an olfactory stimulus (the conditioned stimulus, CS) with sugar reward (the unconditioned stimulus, US) show population-level asymmetries in recalling the sensory stimulus (Letzkus et al., 2006, 2008; Rogers and Vallortigara, 2008; Anfora et al., 2010; Frasnelli et al., 2010a,b; Rigosi et al., 2011; Guo et al., 2016). Specifically, bees in which only one antenna/eye is stimulated by the CS show a dominance of the right side in the ability to recall learned sensory stimuli (Letzkus et al., 2006, 2008; Anfora et al., 2010; Frasnelli et al., 2010a; Rigosi et al., 2011), as do bees trained with both antennae in use and tested for short-term memory (Rogers and Vallortigara, 2008; Frasnelli et al., 2010b). A leftside dominance has been reported for the recall of long-term olfactory memory when bees are trained with both antennae in use (Rogers and Vallortigara, 2008; Frasnelli et al., 2010b). In the case of the olfactory system, population-level asymmetries are already present at the level of the antennae and the antennal lobes, with the right side showing a higher number of antennal olfactory sensilla and a higher separation between odors when antennal lobe responses are evaluated using calcium imaging (Letzkus et al., 2006; Frasnelli et al., 2010a; Rigosi et al., 2015). Also, an increased protein-coding gene expression is observed 24 h after olfactory learning in bees trained with their right antennae as compared with bees trained with only the left ones (Guo et al., 2016).

Although lateralization of olfactory processing might be sufficient per se to trigger the lateralized behavior in the framework of olfactory learning and memory, the contribution of the US, i.e., the sucrose reward, has been largely overlooked. This is particularly surprising as sucrose perception in olfactory conditioning starts at the level of bilateral organs such as the antennae (Esslen and Kaissling, 1976; de Brito Sanchez et al., 2005) and the tarsi (de Brito Sanchez et al., 2014). It has been shown that food rewards trigger an asymmetrical expression of the immediate early-gene c-jun transcript in the honeybee (McNeill and Robinson, 2015; McNeill et al., 2016). Additionally, sucrose responsiveness affects learning and odor discrimination performance in worker honeybees (Scheiner et al., 2001, 2003), so that any left–right bias in sucrose processing might indeed contribute to the observed behavioral asymmetries in olfactory learning and memory in this insect.

Here, we investigated lateralization of sucrose sensitivity of honeybees, both at the level of the antennae and the tarsi. Using the appetitive response of proboscis extension response (PER) of bees to sucrose solutions of increasing concentrations, we compared sucrose responsiveness at the level of left vs. right antennae and tarsi. We report the first evidence for lateralization of sugar sensitivity in the honeybee, with higher responsiveness and resistance to habituation on the right antenna compared to the left one. We discuss the consequences of this lateralization and propose further research avenues in the study of gustatory lateralization in bees.

# MATERIALS AND METHODS

## Animal Preparation

Experiments were carried on using forager bees (Apis mellifera ligustica) caught at a feeder made available each morning at the experimental apiary of the Research Center on Animal Cognition, located in the campus of the University Paul Sabatier (Toulouse, France). Each experimental day, bees were brought to the laboratory, cold anesthetized until immobility (approximately 4–5 min), and harnessed individually within a metal tube using adhesive tape placed in between the head and the thorax. Lowtemperature melting wax was used to further immobilize the head (Matsumoto et al., 2012). Bees used in the antennalresponsiveness assay were prepared as follow: two thin strips of adhesive tape (∼4 mm × 5 cm) were joined together on their sticky side and applied to one of the two antennae as shown in **Figure 1A**. This allowed to block one antenna without damaging it during the first experimental phase in which sucrose

FIGURE 1 | (A) A harnessed honeybee prepared for the antennal responsiveness assay with the upper part of the right antenna (flagellum) blocked by a strip of tape (the tape in contact with the antenna is not sticky) in order to prevent any movement and stimulus detection by this antenna during testing. The left antenna is free to move and can be easily reached by the experimenter to test its sucrose sensitivity. (B) A harnessed honeybee prepared for the tarsal responsiveness assay with its fore-legs fixed wide open in order to allow tarsal gustatory stimulation.

responsiveness via the contralateral antenna was recorded, and to free it in a second experimental phase to assess sucrose responsiveness while the contralateral antenna was blocked in the same way. PER can be elicited in bees immobilized in this way by gently touching the free antenna with a toothpick soaked with sucrose solution. Bees used for tarsal responsiveness were mounted in the metal tubes with fore-tarsi protruded and fixed wide open in order to facilitate their stimulation (**Figure 1B**). PER can be elicited in these bees by touching the left or the right foretarsus with a toothpick soaked with sucrose solution (de Brito Sanchez et al., 2014). Once harnessed, each bee was checked for intact PER and was fed with 5 µl of sucrose solution (50% w/w) to equalize the level of hunger across individuals. After feeding, bees were kept resting for 2 h in a dark and humid place (∼60%) at 25 ± 1 ◦C before proceeding with the experiment. Bees that did not show the reflex were discarded.

#### Sucrose Responsiveness Assay

Two hours after resting, sucrose responsiveness was quantified by recording PER in response to increasing concentrations of sucrose, following a standard protocol (Pankiw and Page, 1999; Scheiner et al., 1999; Scheiner et al., 2003). Sucrose solutions were prepared using sucrose of analytical grade (Sigma-Aldrich, France) diluted in purified water (Milli-Q System, Millipore, Bedford, MA, United States). Each bee was presented with seven sucrose solutions of increasing concentrations: 0.1, 0.3, 1, 3, 10, 30, and 50 (w/w), which were delivered on the free antenna or tarsus with the help of one toothpick. In the case of tarsal stimulation, the tarsus was approached from below to avoid any accidental contact with the antennae and care was taken to ensure that the toothpick contacted both the tarsus and the claws. In the case of antennal stimulation, the antenna was approached from below to minimize the interference with the visual system of the insect and the antenna touched on its middistal part including the tip. In both cases, successive sucrose stimulations were interspersed with purified water stimulations to avoid sensitization. The inter-stimulus interval (either for sucrose or water) was ∼2 min. Bees that did not respond to any sucrose concentration, including the 50%, were excluded from successive analyses (Scheiner et al., 1999). We also discarded bees responding to water to control for the effect of thirst on sucrose responsiveness and those exhibiting inconsistent responses to sucrose (i.e., responding to a lower but not to a higher sucrose concentration) as preconized by the standard method of sucrose responsiveness evaluation (Pankiw and Page, 1999; Scheiner et al., 1999, 2003). To test for lateralization in sucrose responsiveness, the PER assessment to all sucrose concentrations was repeated twice for each bee, one on each side (left vs. right). To balance out the possible effect of testing order, half of the bees were first stimulated with sucrose on their right antenna/tarsus and then on their left antenna/tarsus, while the other half was subjected to the inversed sequence.

The two sequences of stimulation were spaced by 2 h. In the case of bees tested for antennal responsiveness, the tape covering the antenna was moved to the other antenna soon after the end of the first assay. Bees were then fed again with 5 µl of sucrose solution, kept resting for 2 h in a dark and humid place (∼60%) at 25 ± 1 ◦C before proceeding with the second phase of the experiment. For each bee retained for the analysis (antennal sensitivity: n = 101, tarsal sensitivity: n = 88), an individual sucrose response score (SRS) was calculated as the number of sucrose concentrations eliciting a PER (e.g., SRS = 4 for an individual responding to 3, 10, 30, and 50% sucrose solution but not to lower concentrations). SRS ranged from 1 (bees responding only to the 50% sucrose solution delivered at the end of the sequence) to 7 (bees responding to all seven concentrations). For each bee two different SRSs were calculated, one for the left antenna/tarsus and one for the right antenna/tarsus.

#### Non-associative Learning Assay

A subset of bees tested for left–right antennal lateralization (n = 57) was then trained following a habituation protocol to investigate the possible existence of lateralization in habituation to antennal sucrose stimulation. These bees were the last ones tested in the sucrose responsiveness assay. At the end of the previous assay, bees were fed ad libitum and kept resting overnight in a dark and humid place (∼60%) at 25 ± 1 ◦C with both antennae free to move. The day after, bees were fed again with 5 µl of sucrose solution (50% w/w) and one antenna was blocked as explained above. After 2 h resting, bees were subjected to the habituation assay, which consisted of 30 successive stimulations with 10% sucrose solution on the free antenna. Stimulations lasted less than a second and the interstimulus interval was 10 s (Scheiner, 2004; Baracchi et al., 2017). Once the first habituation phase was finished, the bees had a resting period of 2 h. The habituated antenna was blocked and the non-habituated one was released to perform the second habituation phase in the same way. The same right–left or left–right order was used in the sucrose responsiveness and habituation assays so that if a bee was first tested for sucrose responsiveness on the right antenna and then on left antenna (Right 1 and Left 2, n = 27), it was first habituated on the right antenna and then on the left antenna and vice versa (Left 1 and Right 2) (n = 30).

At the end of each habituation phase, a dishabituation trial (DT) was performed 10 s after the last habituation trial. It consisted of a single stimulation with a 50% sucrose solution delivered to the selected antenna. Ten seconds after the DT, the bees received a test stimulation on the same antenna with the original stimulus used in the habituation phase (10% sucrose solution). In all cases, PER (yes/no) to the stimulating solution was assessed. For each antenna, an individual habituation score (HS) was calculated as the number of sucrose stimulations eliciting a PER in the habituation phase. HSs ranged, therefore, from 1 to 30.

#### Data Analysis

Proboscis extension responses (1 or 0) to sucrose stimulation of individual bees in both the sucrose responsiveness and the habituation assays were examined using generalized linear mixed models (GLMMs) with a binomial error structure – logit-link function – glmer function of R package lme4 (Bates et al., 2014). For the sucrose responsiveness assay, either for the tarsal or

the antennal experiment, "Response" was entered as a dependent variable, "Side" and "Order" were entered as fixed factors, and "Sucrose concentration" was entered as a covariate. For the habituation assay, "Response" was the dependent variable, 'Order' was a fixed factor, and 'Trial' was entered as a covariate.

Left–right differences in SRS, either at the level of the tarsi and the antennae, were analyzed with a linear mixed model (LMM). The "SRS" was the dependent variable, the "Side" and the "Order" were fixed factors. Left–right differences in antennal HSs were analyzed with a LMM where the "HS" was entered as a dependent variable, "Side", "Order," and "SRS" were entered as fixed factors. In all models, "Individual" identity (ID) was considered as a random factor in order to allow for repeated measurements. In all cases, we retained the significant model with the highest explanatory power (i.e., the lowest AIC value). The interaction Side <sup>∗</sup> Order was evaluated in all the full models but was not significant in all cases and was, therefore, not included in the selected models. Left–right differences in the response to the DT were tested with a Wilcoxon test, while dishabituation responses to the original stimulus used for habituation were tested with χ 2 test. All statistical analyses were performed with R 3.2.3 (R Core Team, 2016).

#### RESULTS

Using the PER to sucrose solutions of increasing concentration, we investigated lateralization of sucrose sensitivity at the level of the antennae and the tarsi in two independent groups of bees. As expected, PER of harnessed bees increased with sucrose concentrations in both the group of bees tested on the antennae and the one tested on the tarsi of the fore-legs (**Figure 2**; GLMM, Sucrose concentration: antennae: χ <sup>2</sup> = 153.4, df = 1, n = 101, p < 0.0001; tarsi: χ <sup>2</sup> = 241.8, df = 1, n = 88, p < 0.0001). Yet, differences in the patterns of responsiveness were found when comparing antennal and tarsal sucrose responsiveness. Bees tested on the tarsi showed the same level of responsiveness on both sides (GLMM, Side: χ <sup>2</sup> = 2.69, df = 1, n = 88, p < 0.10) but the order of testing affected sucrose responsiveness: in the second stimulation phase, bees responded significantly more, irrespectively of the tarsal side considered (GLMM, Order: χ <sup>2</sup> = 6.04, df = 1, n = 88, p = 0.014, **Figure 2A**). Accordingly, the tarsal SRS, which provides an individual assessment of sucrose responsiveness, was statistically similar on the left and the right side (mean SRS ± SEM: mean right 1 and 2 side 2.6 ± 0.06; mean left 1 and 2 side 2.5 ± 0.07; LMM, χ <sup>2</sup> = 1.08, df = 1, n = 88, p = 0.29) but differed between stimulation phases (**Figure 3A**; LMM, χ <sup>2</sup> = 10.47, n = 88, p = 0.001).

In the case of bees tested on the antennae, we found a lateralization of sucrose sensitivity when comparing the left and right sides (**Figure 2B**; GLMM, Side: χ <sup>2</sup> = 19.30, df = 1, n = 101, p < 0.0001). In particular, the right antenna was more sensitive to sucrose than the left one. This was particularly visible for intermediate sucrose concentrations such as 3% but not for the highest concentration where all groups showed maximal responsiveness (left–right: χ 2 test: 0.1%: p = 0.09; 0.3%: p = 0.32; 1%: p = 0.07; 3%: p = 0.02; 10%: p = 0.07; 30%: p = 0.12), irrespectively of the stimulation phase considered (GLMM, Order: χ <sup>2</sup> = 2.71, df = 1, n = 101, p = 0.1). The SRS analysis confirmed the antennal lateralization detected at the population level (**Figure 3B**; mean SRS ± SEM: right 1 and 2 side 3.9 ± 0.08; left 1 and 2 side 3.3 ± 0.08; LMM, χ <sup>2</sup> = 14.70, n = 101, p = 0.00012) and that the order of testing had no effect (LMM, χ <sup>2</sup> = 1.86, p = 0.17). Thus, while the fore-tarsi did not show evidence of lateralization, the antennae showed a clear asymmetry in sucrose responsiveness.

This asymmetry led us to investigate antennal lateralization of habituation to antennal sucrose stimulation in a subset of bees previously tested for left–right antennal lateralization (n = 57). Bees were first habituated on one antenna and afterward on the other antenna. In both phases, the repeated stimulation with 10% sucrose solution led to significant habituation along trials as PER decreased significantly from the 1st to the 30th habituation trial (**Figure 4**; GLMM, trial: χ <sup>2</sup> = 683.2, df = 1, p < 0.0001). Yet, the degree of PER habituation differed between the left and the right antenna (**Figure 4**; GLMM, Side: χ <sup>2</sup> = 29.78, df = 1, p < 0.0001) and was independent of the order of testing (GLMM, Order: χ <sup>2</sup> = 0.33, df = 1, p = 0.56). Consistently with the higher sensitivity of the right antenna found in the prior experiment, we found that the right antenna was also more resistant to habituation than the left one at the population level, irrespectively of the order of testing. Accordingly, a higher HS was found for the right antenna (i.e., less habituation) compared to the left antenna (**Figure 5**; mean HS ± SEM: right 1 and 2 side, 21.4 ± 0.56; left 1 and 2 side, 17.5 ± 0.64; LMM, χ <sup>2</sup> = 11.30, df = 1, p = 0.001). The SRS of each antenna had indeed a main effect on its HS, thus showing that higher sucrose sensitivity resulted in more resistance to habituation (LMM, χ <sup>2</sup> = 33.86, df = 1, p < 0.0001).

The recovery of PER after replacing the 10% habituation sucrose solution by a 50% sucrose solution (**Figure 4**; dishabituation trial, DT) ruled out that the observed decrease of PER to successive stimulations was due to fatigue and/or sensory adaptation. Indeed, a significant increase of PER was observed between the response in the last habituation trial and in the DT in all cases (**Figure 4**; Wilcoxon test, left 1: n = 30, Z = −4.79, p < 0.001; left 2: n = 27, Z = −4.58, p < 0.001; right 1: n = 27, Z = −4.36, p < 0.001; right 2: n = 30, Z = −4.47, p < 0.001). Stimulating with the original habituating stimulus (10% sucrose solution) in the final test after the DT showed response recovery following dishabituation; in all cases, responses recorded in this last test were significantly higher than those recorded in the last habituation trial (**Figure 4**; Wilcoxon test, all groups p < 0.001). Significant left–right differences were neither found in dishabituation nor in the final test (Wilcoxon test, DT: Z = −0.81, p = 0.41; test: Z = −0.90, p = 0.36).

#### DISCUSSION

In the present work, we studied for the first time gustatory lateralization in the honeybee by testing their sucrose sensitivity both at the level of the antennae and the distal segments of the fore-legs (tarsi). We found that a left–right asymmetry in sucrose sensitivity exists at the level of the antennae. Bees exhibited a

FIGURE 2 | Left–right tarsal (fore-tarsi) (A) and antennal (B) responsiveness to sucrose solution. Both graphs show cumulative percentages of bees showing PER when stimulated with seven sucrose solutions of increasing concentration (0.1, 0.3, 1, 3, 10, 30, and 50% w/w). Approximately half of the bees were tested first on the right antenna (Right 1) and then on the left antenna (Left 2) (n = 49) and vice versa (Left 1 and Right 2) (n = 52). Similarly, about half of the bees were tested first on the right tarsus (Right 1) and then on the left tarsus (Left 2) (n = 51) and vice versa (Left 1 and Right 2) (n = 37). (A) No lateralization of tarsal sucrose sensitivity was found (GLMM, p = 0.10), but a significant effect of the stimulation sequence was detected (GLMM, p = 0.014), with sensitivity being increased during the second stimulation phase. (B) A lateralization of antennal sucrose sensitivity was found at the population level, with the right antenna being significantly more sensitive to sucrose than the left one (GLMM, p < 0.0001).

FIGURE 3 | Left–right tarsal (fore-tarsi) (A) and antennal (B) individual sucrose response scores (SRS). Median, quartiles, and max and min (upper and lower whiskers) SRS values of bees stimulated with seven sucrose solutions of increasing concentration (from 0.1 to 50% w/w) on the left (reddish) and the right (cyan) antenna and tarsus. Black dots represent individual bees. For each bee, SRS values could range between 7 (a bee responding to all seven concentrations) and 1 (a bee responding only to the highest concentration of 50%). (A) No lateralization of sucrose sensitivity was found at the level of the tarsi (LMM, n = 88, p = 0.29) while the order of testing had a significant effect (LMM, p = 0.001). (B) SRS revealed a lateralization of sucrose sensitivity (LMM, n = 101, p = 0.0001) while the order of testing had no effect (LMM, p = 0.17).

higher responsiveness to intermediate sucrose concentrations on the right than on the left antenna, an effect that was independent of the order of stimulation. This asymmetry was also visible in a habituation experiment, where repeated stimulation with an intermediate sucrose concentration on the more sensitive right antenna determined less habituation than on the left antenna. No lateralization was found at the level of the foretarsi where, on the contrary, enhanced responsiveness was found on the second phase of sucrose stimulation, irrespectively of tarsal side. In this case, the successive experience with sucrose seemed to enhance the sensitivity of the bees for both tarsi.

Recall of olfactory memory is lateralized in honeybees as they achieve better retention performances when the odorant acting as CS is delivered to their right rather than to their left antenna after olfactory PER conditioning with single antenna in use (Frasnelli et al., 2014; Guo et al., 2016). Interestingly, when the odorant is delivered to both antennae at the same time during training and presented to single antennae during test, the memory recall is achieved better with the right antenna only 1–2 h after

FIGURE 4 | Left–right antennal habituation to sucrose solution stimulation. Like in the sucrose responsiveness assays, approximately half of the bees were first tested for habituation on the right antenna (Right 1) and then on the left antenna (Left 2) (n = 27) while the other half experienced the reversed sequence (Left 1 and Right 2; n = 30). The two sequences of side stimulation were spaced by 2 h. Habituation consisted in 30 consecutive stimulations with a 10% (w/w) sucrose solution on the free antenna (while the other one was blocked). Ten seconds after the last habituation trial, bees were stimulated on the habituated antenna with a 50% (w/w) sucrose stimulation to induce dishabituation ("dishabituation trial" or DT). Ten seconds after the DT, bees were stimulated on the same antenna with the original stimulus used during the training (i.e., 10% sucrose solution) to check for typical response recovery following dishabituation ("Test"). The right antenna was more resistant to habituation than the left one at the population level (GLMM, p = 0.006). Habituation to sucrose stimulation was significantly affected by the SRS of individual bees (GLMM, p < 0.0001), thus demonstrating that the left–right antennal asymmetry in sucrose sensitivity translates directly into a lateralization of habituation to sucrose stimulation. The order of testing had no effect on habituation. No significant left–right differences in the DT as well as in the test were observed (Wilcoxon test, DT: p = 0.41, test: p = 0.36). The DT as well as re-stimulating with the original stimulus induced a significant response recovery, which did not differ between sides. This demonstrates that the observed decrease in PER to the 10% sucrose solution was a real case of habituation and was not due to sensory adaptation or fatigue.

training, while at 6–23 h the recall is better performed with the left antenna (Rogers and Vallortigara, 2008). The mechanisms underlying this asymmetry remain to be clarified but, most likely, they can be partially retraced to left–right differences both at the peripheral and at the central level. At the peripheral level, olfactory sensilla (i.e., sensilla placodea, trichodea, and basiconica) have been found to be more abundant on the right than on the left antenna, suggesting that lateralization in memory retrieval may arise from asymmetries in the detection of the conditioned odor stimulus (CS) during appetitive olfactory training (Letzkus et al., 2006; Frasnelli et al., 2010a). At the central level, neural responses in the left and right antennal lobes differ, so that odor encoding in these structures results in higher separation (i.e., better discriminability) in the right antennal lobe (Rigosi et al., 2015). Moreover, 24 h after olfactory PER conditioning with single antennae, the right side of the brain shows increased gene-expression compared to the left one (Guo et al., 2016).

These findings clearly underline that asymmetries in olfactory retrieval have a correlate at various levels of odorant (CS) processing in the bee nervous system. However, these asymmetries might also correlate with additional asymmetries at the level of gustatory (US) processing. In olfactory PER

FIGURE 5 | Left–right antennal habituation score (HS). Median, quartiles, and max and min (upper and lower whiskers) sucrose response values of individual HSs for bees subjected to two phases of 30 consecutive antennal stimulations with 10% sucrose solution. Approximately half of the bees were tested for habituation first on the right antenna (Right 1) and then on the left antenna (Left 2) (n = 27) and vice versa (Left 1 and Right 2) (n = 30). Black dots represent individual bees. Bees with a score of 30 responded to all 30 sucrose stimulations, i.e., did not show any habituation. The right antenna had a higher score than the left antenna (LMM, p = 0.001) indicating a higher resistance to habituation on the right side compared to the left one. The order of testing had no significant effect on the HS.

conditioning, sucrose is the US used to induce PER. In the first versions of this protocol, sucrose was delivered to the fore-tarsi and then to the proboscis (Takeda, 1961) while in the more recent and standard protocol, it is first delivered to the antennae and then to the proboscis (Matsumoto et al., 2012). Among these gustatory appendages, only the antennae exhibited a differential sucrose sensitivity between the left and the right side. However, it worth noting that this asymmetry was only visible for sucrose concentrations that are typically not used in olfactory PER condition as they are too low (e.g., 3%) to support efficient learning (Matsumoto et al., 2012). The same remark may apply to other conditioning protocols. For instance, in a recently established gustatory conditioning protocol, bees receive tastants on the antennae and, afterward, a mild electric shock which induces the sting extension reflex (SER) (Guiraud et al., 2018). Over the successive trials, bees learn to extend the sting to aversive tastes. No left–right antennal asymmetries were found when bees learned the association between sucrose and the shock. Yet, the concentration of sucrose (33%) used in this protocol (Guiraud et al., 2018) falls within the range in which asymmetries were no longer evident in our work. Interestingly, contrary to olfactory PER conditioning, in gustatory SER conditioning, sucrose can be used at concentrations lower than 33% as it does not represent the US but the CS (Guiraud et al., 2018). As sucrose sensitivity is a crucial determinant of learning performance in an associative learning task, we predict that the antennal asymmetry in sucrose sensitivity revealed by our work is likely to translate into an asymmetrical performance in aversive gustatory learning and/or memory formation when low sucrose concentrations are used as CS.

Asymmetric performances during olfactory learning between bees with only their left or their right antenna in use have been reported only for learning to detect an odorant from a background but not during differential odor learning (Rigosi et al., 2015). Interestingly, we demonstrated that a left–right asymmetry exists in a simple form of non-associative learning (habituation). As expected, the observed lateralization in sucrose responsiveness (i.e., higher sucrose sensitivity on the right antenna) corresponded to a lateralization in the same direction in habituation to successive sucrose stimulations (i.e., more resistance to habituation upon stimulation on the right antenna). This finding is not surprising given the correlation existing between sucrose responsiveness and habituation to antennal sucrose stimulation in bees (Scheiner, 2004).

The mechanisms underlying left–right sucrose responsiveness asymmetries in the antennae may also involve left–right asymmetries at the peripheral level, i.e., in the gustatory sensilla/receptors located on these appendages. Analyses of nonolfactory sensilla located on the bee antennae, which included a category of gustatory sensilla (i.e., sensilla chaetica), found slightly more sensilla in the segments 3rd–9th of the left than on the right antenna (Frasnelli et al., 2010a). This asymmetry does not align with our finding that bees are more sensitive to sucrose on the right antenna. However, when the last distal segment of the flagellum (10th segment) which constitutes the primary antennal contact region was considered, the situation was reversed with slightly more non-olfactory sensilla on the right than on the left antenna (Frasnelli et al., 2010a). Importantly, sensilla chaetica, which are responsible for sucrose detection and respond in a dose-dependent manner to sucrose solution (Haupt, 2004; de Brito Sanchez et al., 2005), show a high degree of variability in spike frequency and sucrose response within the same antenna (Haupt, 2004). Lateralization of sucrose responsiveness might be, therefore, due to left–right differences in the proportion of distal sensilla chaetica with different sensitivities to sucrose rather than to differences in their absolute number.

Besides the antennae, other body regions such as the subesophagic zone (SEZ) of the brain might be involved in the observed left–right lateralization. The SEZ is the main central gustatory area in the insect brain (see de Brito Sanchez, 2011). It has a major role in gustatory encoding but also participates in the motor control of mouthparts and mechanosensory information processing. Contrary to the olfactory system, which is organized in two distinct brain hemispheres, the SEZ is a central unpaired brain area. Several sucrose processing neurons with their soma located in the ventral and median region of the SEZ (VUM neurons: ventral unpaired median neurons) and arborizing within different regions of the bee brain have been reported (Schröter et al., 2006). Whether, despite its unpaired nature, the SEZ presents a lateralized gustatory processing contributing to the observed antennal gustatory asymmetries remains to be determined. Interestingly, gustatory receptor neurons hosted by antennal gustatory sensilla do not only project to the unpaired SEZ (Pareto, 1972; Suzuki, 1975; Haupt, 2007) but also to two adjacent lateral regions termed the lateral lobes (one on each side of the SEZ) (Haupt, 2007). Left–right asymmetries in antennal sucrose processing may have, therefore, a neural correlate at the level of the lateral lobes with enhanced signaling on the right side compared to the left side. This possibility remains so far unexplored and further investigations will be necessary to understand the neural underpinnings of lateralization in the case of antennal sucrose responsiveness.

The adaptive value of the lateralized sucrose sensitivity at the level of the antennae remains unclear. From an ecological perspective, it would be interesting to determine whether foragers entering in contact with food, be it pollen or nectar, exhibit some bias prioritizing a first contact with the right antenna. As the right antenna is particularly sensitive to low sucrose concentrations (e.g., 3%), which correspond to those found in various pollen types (Szcze˛sna, 2007), pollen contact may be lateralized similarly to what occurs at the level of social interactions. In this case, bees display a lateral preference to use their right antenna in positive interactions with other bees involving food exchange (Rogers et al., 2013). Moreover, an antennal bias in sugar responsiveness could help optimizing side-specific odor memory formations and retention during foraging. Specializing the right side for immediate-short-term odor memory has been hypothesized to be of aid for building odor memories in a more efficient way to reduce interference of two types of neural processing (learning and recalling) during foraging activity over time (Rogers and Vallortigara, 2008). When bees are trained in the lab with both antennae in use, the recall of odor memories shows a shift of antenna dominance over time, with the right antenna specialized for short-term memory recall and the left one for long-term memory recall (Rogers and Vallortigara, 2008; Frasnelli et al., 2010b). Having the left antenna less tuned to sugar responsiveness could contribute to this specialization, leaving the left side free to perform a parallel task possibly tuned to long-term memory formation.

Similarly to what has been reported by previous works (Marshall, 1935; de Brito Sanchez et al., 2008), we found that the fore-tarsi were less sensitive to sucrose than the antennae, a fact that may be related to the different number of taste sensilla located on these gustatory structures [antennae count about 15–30 times more receptors than the tarsi (Whitehead and Larsen, 1976; de Brito Sanchez, 2011)]. Contrary to the case of the antennae, no evidence for lateralization of sucrose sensitivity was found at the level of the distal segments of the fore-tarsi. Each tarsomere has two types of gustatory sensilla, 10– 21 sensilla chaetica and 0–6 sensilla basiconica (Whitehead and Larsen, 1976) but whether these numbers differ between the left and right fore-tarsi remains to be determined. Similarly, whether projections of the gustatory receptors hosted by these sensilla to the central level (i.e., to the thoracic ganglion and eventually to the SEZ) differ between sides is unknown. Our behavioral results do not seem to support the existence of differences in the number and/or sensitivity of gustatory receptors at either the peripheral or the central level.

An interesting finding concerning tarsal sucrose sensitivity was the significant effect of sequence stimulation. When bees were stimulated with increasing sucrose concentrations on one tarsus, sucrose sensitivity was increased in the contralateral tarsus, irrespective of the side considered. This result indicates that excitation induced by sucrose stimulations is transferred via

central integration to the contralateral tarsus. Previous behavioral experiments showed that sucrose solution delivered on one tarsus elicits immediate PER which cannot be inhibited by any other aversive substance delivered afterward on the contralateral tarsus. In these conditions, sucrose was suggested to act as a "winner takes-all" stimulus, suggesting "a process of central integration, probably at the level of the thoracic ganglion" (de Brito Sanchez et al., 2014). This hypothesis is consistent with the present findings and indicates that when bees detect sucrose with one fore-tarsus, they become "prepared" to sense sucrose with the opposite tarsus, a mechanism that may serve efficient location of minute nectar sources. Importantly, this effect cannot be attributed to sensitization, the enhancement of responsiveness due to non-associative experience with a repeated biologically relevant stimulus like food (Squire and Kandel, 1999). In the honeybee, sensitization is only observable after very short intervals (seconds to few minutes) following food stimulation (Menzel, 1999). The fact that an interval of 2 h was interspersed between the two sucrose stimulation phases excludes the possibility of bees being sensitized by the first stimulation phase. Moreover, given the fact that the antennae are more sensitive to sucrose than the tarsi (see above), if sensitization would have occurred, it should have been observed at the level of the antennae rather than at the level of the tarsi. This was not the case and rules out, therefore, the possibility of sensitization accounting for enhanced responsiveness between fore-tarsi.

To date, clear anatomical asymmetries at the level of the brain are still lacking for honeybees (Haase et al., 2011a,b; Rigosi et al., 2011) and differences in the number or sensitivity of olfactory and non-olfactory sensilla are unlikely to explain

#### REFERENCES


entirely the behavioral laterality found in this insect (Frasnelli et al., 2012). Phenomena such as the lateral shift (Rogers and Vallortigara, 2008) or the side specificity of olfactory learning and generalization (Sandoz and Menzel, 2001; Sandoz et al., 2002) together with evidence of asymmetry in gene expression (Biswas et al., 2010; McNeill and Robinson, 2015; Guo et al., 2016; McNeill et al., 2016) and odor processing (Rigosi et al., 2015) described in honeybees suggest, indeed, that asymmetries at the central level also exist and await for better characterizations.

## AUTHOR CONTRIBUTIONS

ER, GdBS, and MG conceived the study. DB designed the experiments, performed the experiments with the help of students, and carried out the data analysis. All authors contributed equally to the writing of the manuscript.

# FUNDING

This work was supported by the Centre National de la Recherche Scientifique (CNRS) and Paul Sabatier University of Toulouse.

# ACKNOWLEDGMENTS

Thanks are due to Lucie Hotier for help with collecting bees and to Gwenaïs Templier for help with data collecting. ER thanks the support of the Wenner-Gren Foundation.



**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2018 Baracchi, Rigosi, de Brito Sanchez and Giurfa. This is an openaccess article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Odor Learning and Its Experience-Dependent Modulation in the South American Native Bumblebee Bombus atratus (Hymenoptera: Apidae)

#### Florencia Palottini1,2† , María C. Estravis Barcala1,2† and Walter M. Farina1,2 \*

<sup>1</sup> Laboratorio de Insectos Sociales, Departamento de Biodiversidad y Biología Experimental, Facultad de Ciencias Exactas y Naturales, Universidad de Buenos Aires, Buenos Aires, Argentina, <sup>2</sup> Instituto de Fisiología, Biología Molecular y Neurociencias (IFIBYNE), CONICET – Universidad de Buenos Aires, Buenos Aires, Argentina

#### Edited by:

Lars Chittka, Queen Mary University of London, United Kingdom

#### Reviewed by:

Jean-Marc Devaud, Université Toulouse III – Paul Sabatier, France Johannes Spaethe, Universität Würzburg, Germany Ellouise Leadbeater, Royal Holloway, University of London, United Kingdom

#### \*Correspondence:

Walter M. Farina walter@fbmc.fcen.uba.ar; walter@bg.fcen.uba.ar

†These authors have contributed equally to this work.

#### Specialty section:

This article was submitted to Comparative Psychology, a section of the journal Frontiers in Psychology

Received: 25 January 2018 Accepted: 10 April 2018 Published: 27 April 2018

#### Citation:

Palottini F, Estravis Barcala MC and Farina WM (2018) Odor Learning and Its Experience-Dependent Modulation in the South American Native Bumblebee Bombus atratus (Hymenoptera: Apidae). Front. Psychol. 9:603. doi: 10.3389/fpsyg.2018.00603 Learning about olfactory stimuli is essential in bumblebees' life since it is involved in orientation, recognition of nest sites, foraging efficiency and food yield for the colony as a whole. To evaluate associative learning abilities in bees under controlled environmental conditions, the proboscis extension response (PER) assay is a well-established method used in honey bees, stingless bees and successfully adapted to bumblebees of the genus Bombus. However, studies on the learning capacity of Bombus atratus (Hymenoptera: Apidae), one of the most abundant native species in South America, are non-existent. In this study, we examined the cognitive abilities of worker bees of this species, carrying out an olfactory PER conditioning experiment. Bumblebees were able to learn a pure odor when it was presented in paired association with sugared reward, but not when odor and reward were presented in an unpaired manner. Furthermore, if the bees were preexposed to the conditioned odor, the results differed depending on the presence of the scent either as a volatile in the rearing environment or diluted in the food. A decrement in learning performance results from the non-reinforced pre-exposure to the to-be-conditioned odor, showing a latent inhibition phenomenon. However, if the conditioned odor has been previously offered diluted in sugared reward, the food odor acts as a stimulus that improves the learning performance during PER conditioning. The native bumblebee B. atratus is thus a new hymenopteran species capable of being trained under controlled experimental conditions. Since it is an insect increasingly reared for pollination service, this knowledge could be useful in its management in crops.

Keywords: bumblebee, associative learning, latent inhibition, odor pre-exposure, Bombus atratus

# INTRODUCTION

Bumblebees of the genus Bombus (Hymenoptera: Apidae) are social insects with an annual life cycle which play an important role as pollinators in natural and agricultural ecosystems. For this reason, presently, their colonies are commercialized to improve the production of diverse crops (Heinrich, 2004). However, the worldwide trade in bumblebee colonies for crop pollination, in particular

**40**

of B. terrestris, has elicited special concern about the potential for invasion by non-native bumblebees and their impacts on native pollinator species (Morales et al., 2013).

Bombus atratus Franklin is present in almost all South American countries, except for northern Brazil, Guyana, and Chile (Abrahamovich et al., 2001). It is the most widely distributed and most abundant bumblebee species in Argentina, with great climatic and altitudinal tolerance (Abrahamovich et al., 2001). Because of a clear evidence about their efficiency to pollinate diverse crops grown under cover as tomatoes, eggplants, sweet peppers, blueberries and kiwifruits; colonies of this native species, as others species of the same genus, are commercialized to improve the plant production in pollination services (Aldana et al., 2007; Basualdo et al., 2013; Godoy et al., 2013; Alvarez et al., 2014; Riano et al., 2015).

Learning about olfactory stimuli is essential in bumblebees' life. In particular, in an appetitive context, when collecting at a flower, bees establish an associative memory between a floral scent and the nectar reward, setting out a contingency between the Conditioned Stimulus (CS, floral odor) and the Unconditioned Stimulus (US, nectar). In this way, associative learning represents the basis for efficient foraging behavior in bees, because it allows them to relocate specific food sources and efficiently collect pollen and nectar from different species of flowers. Indeed, bumblebee's foragers are able to learn the quality (in terms of nectar sugar concentration) of the flowers they visit and subsequently tend to specialize on the more profitable species (Cnaani et al., 2006). Furthermore, bumblebees possess the ability to learn and use memories to discriminate flowers on the basis of diverse floral properties, including morphology, color, scent and nectar quality (Dukas and Real, 1993; Chittka et al., 1997; Gumbert, 2000; Spaethe et al., 2007; Leonard et al., 2011).

Examples that bumblebees modify their performance during the search for food outside the nest if they experienced scented nourishment that circulated inside the colony have been reported previously (Dornhaus and Chittka, 1999; Molet et al., 2009). However, the nature of the behavioral mechanisms involved in the information transfer process is unknown. The exposure to a neutral stimulus paired or not with the unconditioned one before the training process could affect differently the behavioral response toward the stimulus to be conditioned (Mackintosh, 1994). If the experimental subject was previously exposed to a CS without pairing with the US and the acquisition of an association is delayed, this phenomenon is defined as latent inhibition, LI (Lubow and Moore, 1959; Lubow, 1973). Contrarily, previous experiences of the CS paired with the US might act as a stimulus that improves associative learning (Mackintosh, 1994). Non-associative processes could also occur, such as the case of sensory pseudoconditioning, where an increase in the response is observed just by the repeated presentation of reinforcement, or sensory priming, in which a preexposed sensory stimulus such as an odor influences a response to a subsequent stimulus of the same sensory modality (Bouton and Moody, 2004). Thus, the assessment by using a standardized learning protocol with individuals of known experience is a way to determine if the mechanisms involved are of sensory or cognitive nature.

The proboscis extension reflex (PER) is part of the behavior to search for food inside the nectaries and allows worker bees to draw up nectar and pollen from flowers. Under controlled environmental conditions, the PER is a well-established method used in honey bees (Takeda, 1961; Bitterman et al., 1983), stingless bees (Mc Cabe et al., 2007; Mc Cabe and Farina, 2009, 2010) and some species of the genus Bombus (Laloi et al., 1999; Riveros and Gronenberg, 2009; Toda et al., 2009; Sommerlandt et al., 2014; Lichtenstein et al., 2015), that allows researchers to evaluate associative learning abilities. However, until now, the learning capacity of the native South American B. atratus species is unknown.

Bearing this in mind, the present research aimed to examine the cognitive capacity of B. atratus worker bumblebees, performing an olfactory classical PER conditioning procedure. First, we evaluated the bumblebees' ability to associate an odorant cue with reinforcement. Furthermore, pre-exposure protocols were applied to analyze the influence of previous experiences in the learning performances. On the one hand, to evaluate the presence of a latent inhibition phenomenon, we performed an odor pre-exposure in the environment. Finally, in another experiment, we evaluated the effect of the prestimulation with a scented sugar solution with the odor to be used as CS in the classical conditioning.

This is the first report about odor learning abilities in the South American native bumblebee B. atratus.

# MATERIALS AND METHODS

#### Study Site, Animals, and Odorant Cues

Eleven bumblebee colonies (B. atratus Franklin) were provided by Biobest Argentina S.A. (Burzaco, Province of Buenos Aires, Argentina) and maintained in the laboratory at the Experimental Field of the University of Buenos Aires, Argentina (34◦ 320 S, 58◦ 260W). All experiments were carried out during the summerautumn season of 2017 and 2018. The colonies were housed in their original commercial boxes (27 cm × 24 cm × 20 cm). The boxes were kept in the laboratory under natural daylight conditions filtered through window glass and fed ad libitum with a sugar solution provided by the supplier and honey bee-collected pollen.

A pool of seven colonies was used to carry out Experiment 1, while six colonies were allocated to Experiments 2 and 3. To exclude colony effects, individuals of the assigned colonies contributed to the data of the experimental and the corresponding control series within each experiment.

Pure odors commonly presented in the floral fragrances (Knudsen and Tollsten, 1993; Raguso and Pichersky, 1999), such as the case of linalool (LIO), phenylacetaldehyde (PHE) and nonanal (NONA; Sigma-Aldrich, Steinheim, Germany), were used during the experiments.

#### Bees' Capture and Harnessing

Colonies were anesthetized with carbon dioxide and individual workers of unknown age and various sizes (intertegula span between 2.4 and 4.44 mm) were randomly captured and confined

in wooden cages (10 cm × 10 cm × 10 cm) in groups of 10–15 individuals, to reduce the stress level and increase the survival rate (personal observation during bees manipulation). Bees were fed ad libitum with 1.8 M unscented sucrose solution and kept in darkness in an incubator for 2 h at 25◦C and 75% relative humidity.

Experimental bees were then anesthetized and harnessed in metal tubes so that only the antennae and mouthparts could freely move. Bees were fed with 1.8 M unscented sucrose solution and kept in the incubator for 20.5 h under the same conditions previously described, prior to olfactory conditioning (**Figure 1A**). Once the time has passed, a restrained bumblebee was placed individually in front of the device used for application of the odorant during the conditioning protocol.

#### Behavioral Assays

#### Proboscis Extension Response Protocol

Bumblebees underwent a classical conditioning protocol adapted from the proboscis extension response (PER) paradigm, which is well established in honey bee olfactory learning procedure (Takeda, 1961; Bitterman et al., 1983). To assay the PER, a device that delivered a continuous airflow (50 ml/s) was used for the application of the odorant. Four microliters of pure odorant impregnated on 30 × 3 mm filter paper inside a syringe were delivered through a secondary air-stream (6.25 ml/s) to the head of the bee. A fan extracted the released odors to avoid contamination (Fernández et al., 2009). Each learning trial lasted 39 s. Before odor presentation, bees rested for 15 s in the airflow for familiarization as well as for testing the bees' response toward the mechanical stimulus. For the training procedure of the classical conditioning, we presented the CS for 6 s. Reinforcement (1.8 M sucrose solution) was presented on the proboscis (mouthparts) and occurred for 3 s, 3 s after the onset of the CS. Memory retention tests were performed 10–15 min after the last conditioning trial and consisted of the presentation of the CS and of a novel odor (NO), both without reinforcement. We considered the PER during the first 3 s of the presentation of the test odor. The order of presentations of the two odors was chosen at random prior to the onset of the test to avoid possible sequential effects. Thus, half of the subjects were tested with the CS first and the NO second, while the other half, with the reversed sequence. Only bees that did not respond to the mechanical airflow stimulus were used.

#### Experiment 1: Olfactory Classical Conditioning

As an initial approach to study if native bumblebees have the ability to associate an odorant cue with reinforcement, we performed an odor classical conditioning with a pure odor as CS, LIO. A second pure odor was used as novel odor during the testing phase, nonanal (NONA). Bumblebees underwent 10 training trials of paired CS-US presentations. In addition to the paired group, for which the presentation of the CS (LIO) was paired with the US, another group received unpaired presentations of the CS and of the US in a pseudo-randomized sequence, as an explicitly unpaired control group (Matsumoto et al., 2012). Both groups underwent a total of 20 trials. The paired group was subject to 10 training trials of paired CS-US presentations and 10 blank trials in between, in which each bee was placed in the setup without any stimulation for 39 s. Thus, both groups had exactly the same sensory experience (10 CS and 10 US presentations) with an average ITI of 10 min (**Figure 1B**). Retention tests were performed 10 min after the last training trial. Those bees that extended their proboscis in the first trial during the odor presentation (innate response) were excluded and they did not finalize the training protocol.

To determine whether increases in conditioned responses in the absolute conditioning were a consequence of associative learning and did not depend on the odor identity, a different pure odor, PHE, was used as CS in a second series of this experiment following the same protocol described above. In this series, retention tests were performed 15 min after the last conditioning trial and consisted of presentations of the CS and of a novel odor (NONA), both without US.

#### Olfactory Stimulation Before Conditioning

To study the influence of previous odorant experiences in the learning performance at the PER setup in B. atratus bumblebees, harnessed individuals were subjected to volatile pre-exposure in the environment by using the same odor to-be conditioned during the training (in order to evaluate the phenomenon of latent inhibition) (**Figure 1C**) or to a prestimulation with a scented sugar solution (to assess the effect of the odor as preconditioned stimulus) (**Figure 1D**).

#### Experiment 2: Volatile Pre-exposure

To carry out the odor exposure, harnessed bees were moved to another incubator (same conditions of temperature, relative humidity, and darkness). There, bees were placed inside a plastic box (20 cm × 10 cm × 6 cm), where 60 µl of pure odor (LIO) was presented in four filter papers (1.5 cm<sup>2</sup> evaporation surface) located on the sides of the box. To reduce odor accumulation, an air extractor was connected to the incubator. After the odor exposure (1 h), bees were moved back to the first incubator to prevent odor contamination during the non-exposure period before starting the absolute conditioning (30 min). Another group never exposed to the odor was used as control (**Figure 1C**).

#### Experiment 3: Prestimulation With Scented Food

In this case, individual workers were confined in a plastic queen cage. Herein, bees were fed with 20–40 µl of the scented food offered through Multipette <sup>R</sup> M4-Repeater <sup>R</sup> M4. Odor solutions were obtained by mixing 50 µl of pure odorant (LIO) per liter of 1.8M sucrose solution. Another group of bees fed with unscented sugar solution was used as a control. Once fed, bees were harnessed as described above and located in the incubator (odorless condition) until the time of the conditioning (**Figure 1D**).

#### Statistical Analysis

All statistical tests were performed with R v3.3.3 (R Development Core Team, 2016). The PER was assessed by means of generalized linear mixed-effect models (GLMM) following a binomial error distribution and using the glmer function of the lme4 package (Bates et al., 2015). In the case of training,

we considered treatment (a two-level factor corresponding to control or odor; control or preexposed) and trials (a tenlevel factor corresponding to 1–10 trials) as fixed effects, with each bee included as a random factor. In the case of test, we considered treatment (a two-level factor corresponding to control or preexposed) and odor (a two-level factor: CS or NO) as fixed effects, with each bee included as a random factor. GLMM were simplified as follows: significance of the different terms was tested starting from the higherorder terms model using anova function to compare between models (Chambers and Hastie, 1992). Non-significant terms (P > 0.05) were removed (see Supplementary information). We considered the use of GLMM because these models allow analyzing response variables whose errors are not normally distributed, avoiding the transformation of the response variable or the adoption of non-parametric methods (Crawley, 2013).

# RESULTS

# Experiment 1: Olfactory Classical Conditioning

When bees were trained to associate a sucrose reward with LIO as odor stimulus, workers were able to build an association between CS and US after a paired presentation (**Figure 2A**). In the training phase, the proportion of bumblebees responding to the CS increased with successive conditioning trials only in the case of paired group, reaching 51% of conditioned responses at the tenth trial (Minimal adequate model: Response ∼ Treatment + Trial + 1| ind., p < 0.001; Supplementary Table S1). In the testing phase, bumblebees showed a significantly different response between treatments (p < 0.01) and between LIO and the novel odor (p < 0.001; Minimal adequate model: Response ∼ Treatment + odor + 1| ind.; Supplementary Table S1).

between LIO and the novel odor (Minimal adequate model: Response ∼ Treatment + odor + 1| ind.; paired, filled bars; unpaired, emptied bar). In the case of phenylacetaldehyde, bumblebees showed a significantly different response between odors (Minimal adequate model: Response ∼ odor + 1| ind.) Sample sizes are indicated in brackets. Asterisks mean significant differences in the learning performance (p < 0.001).

When a different odor was used as CS, bumblebees also exhibited associative learning (**Figure 2B**). The acquisition curve for PHE was similar to the one obtained when bees were conditioned to LIO (**Figure 2A**). Bumblebees responded significantly more often to the CS odor in the paired than in the unpaired group (p < 0.001), reaching a level of 58% at the tenth trial. The unpaired training group showed negligible levels of response: one bumblebee just responded once at the seventh

adequate model: Response ∼ Treatment + Trial + 1| ind.). In the testing phase, bumblebees showed a significantly different response between treatments and

trial (Minimal adequate model: Response ∼ Treatment + Trial + 1| ind; Supplementary Table S1). In the testing phase, the statistical analysis (GLMM) was only carried out taking into account the paired group because of the lack of response in the unpaired group. Herein, bumblebees presented significantly higher responses to the CS than to the NO (Minimal adequate model: Response ∼ odor + 1| ind, p < 0.001; Supplementary Table S1).

Moreover, in order to rule out the possibility that the results of the memory test were not caused by an insensitivity of the bees to nonanal, we performed the conditioning protocol with this odor as CS and LIO as NO (Supplementary Figure S1).

# Olfactory Stimulation Before Conditioning

#### Experiment 2: Volatile Pre-exposure

**Figure 3** shows the acquisition curve of bees after an olfactory pre-exposure. The statistical analysis showed a significant effect of the interaction between treatment and trial (**Figure 4**; Minimal adequate model: Response ∼ Treatment × Trial + 1| ind; Supplementary Table S1). Then, the simple effect analyses denoted that preexposed bees initially exhibited decreased learning compared with unexposed bees (Trial 1 vs. Trial 2: Control, Z-value = 3.768, p < 0.05; Preexposed, Z-value = 1.149, p = 0.9998). Throughout trials, bees of both groups achieved a high level of response, showing no significant differences in the retention performance (Minimal adequate model: Response ∼ odor + 1| ind., p < 0.001).

#### Experiment 3: Prestimulation With Scented Food

**Figure 4** shows the acquisition and retention performances of individuals exposed or not to LIO. When odor exposure was paired with sucrose reinforcement prior to conditioning, bumblebees exhibited a higher performance throughout trials (Minimal adequate model: Response ∼ Treatment + Trial + 1| ind.). Preexposed individuals showed a high initial level of response (39% of bees that extended the proboscis during the first presentation of the odor), reaching a level of 57% at the tenth trial. On the contrary, the acquisition curve of unexposed bees was similar to the one obtained when bees were conditioned to LIO or PHE (see section "Results"). No such asymmetry was found in the retention performances of both groups. Individuals learned equally, showing a significantly different response between odors but not between treatments (Minimal adequate model: Response ∼ odor + 1| ind., p < 0.001).

# DISCUSSION

Our study demonstrates that the South American native bumblebees B. atratus possess clear abilities to associate an a priori neutral stimulus with reinforcement. We showed that workers of this species, in an olfactory classical PER conditioning protocol can learn a pure odor when it was presented in paired association with a sugar reward, regardless of the odor identity, in this case, LIO or PHE. In addition, when we analyzed the influence of the previous olfactory experiences, bees showed a decrement in learning performance resulting from

FIGURE 3 | Experiment 2: Effects of volatile pre-exposure in bumblebees classical conditioning. Percentage of bees that extended the proboscis as response to the odorant (% PER) during the ten trails in which the conditioned odor was paired with sucrose reward (training, left panel) and bees that responded during a testing period 15 min after training (test, right panel). Bumblebees were exposed (filled circles) or not (emptied circles) to the conditioned odor linalool (CS) before olfactory conditioning. In the first trials, preexposed bumblebees exhibited a lower response (Minimal adequate model: Response ∼ Treatment × Trial + 1| ind.). In the testing phase, bees of both groups responded equally well, showing a significantly different response between odors (Minimal adequate model: Response ∼ odor + 1| ind.; preexposed, emptied bars; control, filled bars). Sample sizes are indicated in brackets. Asterisks mean significant differences in the learning performance (p < 0.001). Nonanal was used as novel odor during the testing phase (NO).

FIGURE 4 | Experiment 3: Effects of scented food in bumblebees classical conditioning. Percentage of bees that extended the proboscis as response to the odorant (% PER) during the ten trails in which the conditioned odor was paired with sucrose reward (training, left panel) and bees that responded during a testing period 15 min after training (test, right panel). Bumblebees were fed either sucrose solution (SS) scented with linalool (CS, filled circles) or unscented sucrose solution (emptied circles) before classical conditioning. In the training phase, learning performance of exposed bumblebees increased significantly (Minimal adequate model: Response ∼ Treatment + Trial + 1| ind.). In the testing phase, bees of both groups responded equally well, showing a significantly different response between odors (Minimal adequate model: Response ∼ odor + 1| ind.; unscented, emptied bars; scented, filled bars). Sample sizes are indicated in brackets. Asterisks mean significant differences in the learning performance (∗∗p < 0.01; ∗∗∗p < 0.001). Nonanal was used as NO.

the non-reinforced pre-exposure in the rearing environment to the to-be-conditioned odor. Nevertheless, when a scented food was administered, workers improved their learning performance during PER conditioning to the known odor. The variability in the learning acquisition curves observed in the different control series could be due to a seasonal, as it was observed in honey bees (Lehmann et al., 2011), or to a colony effect. To avoid the first situation we performed control groups for each experimental series corresponding to the different experimental series. To discard the latter situation, we ensured that multiple colonies were used in each experiment and both treatments were assigned to the allocated colonies. Moreover, B. atratus individuals were able to perceive and learn the odor used as novel odor during the all experimental series, Nonanal, ruling out a possible asymmetry odor perception.

The ability of bumblebees to associate a specific odor with a sucrose solution constitutes the basis for learning that certain flowers provide nectar rewards and, consequently, for identifying the most profitable food resources. In this respect, we showed that B. atratus workers can establish this association, reaching a level of more than 50% correct responses after ten training trials. Our results are consistent with those reported in B. terrestris by Sommerlandt et al. (2014) (ca. 60%), but not with Laloi and Pham-Delègue (2004) (ca. 30%). Furthermore, our results differ from Riveros and Gronenberg (2009) whose study was performed on B. occidentalis (ca. 85%). This variable learning performance in bumblebees could be due to the different methodologies carried out, as a different intertrial interval (ITI) during conditioning or hours spent in the incubator (Toda et al., 2009).

When we set out to evaluate the influence of previous olfactory experiences in the learning performance of bumblebees workers B. atratus, we found dissimilar effects depending on the presence of the scent either as a volatile in the rearing environment (without pairing with the unconditioned stimulus, US) or diluted in the food (associated with the reward). Our results showed that olfactory exposure in the environment 1.5 h prior to conditioning, delayed the establishment of a predictive relationship between the exposed odorant and the reward during a later PER conditioning procedure, as a consequence of a latent inhibition effect (as in honey bees, Chandra et al., 2000; Fernández et al., 2009). This, defined by Lubow (1973), is a phenomenon in which the first-learning information interferes with memory for the second-learning one. Thus, it makes that subjects that have been preexposed to a CS without reinforcement delay the conditioned response when the CS is paired with the US. This is the first report about the presence of latent inhibition in bumblebees. In contrast with our results, other studies found the occurrence of sensory priming in bees, a non-associative phenomenon, after an odor pre-exposure (Molet et al., 2009; Roselino and Hrncir, 2012). Roselino and Hrncir (2012), working with stingless bee foragers Melipona scutellaris, found that repeated, albeit unrewarded, presentation of an odor significantly influenced the subsequent food choice of foragers, biased toward the preexposed odor. Likewise, Molet et al. (2009) showed that the presence of a floral scent in the nest environment

in the absence of a reward is itself sufficient to bias the landing preference of B. terrestris, even if the exposure time is short, suggesting that bees either had learned the volatile scent or had been sensory-primed by perceiving it. Since these authors did not prevent the possible contact with honeypots inside the nest, the association of the odor and the honey reward could not be ruled out.

On the other hand, when odor exposure was paired with sucrose reinforcement prior to conditioning (20.5 h beforehand), bumblebees increased their responses to the CS during trials, due to the fact that food odor acts as a previous stimulus (current study). Our results are consistent with those reported by McAulay et al. (2015), who demonstrated that contacts with scented food inside the B. impatiens nest, increased the likelihood a bee would respond to the scent. Even more, individuals that failed to contact a honeypot containing the scented sucrose solution exhibited no response to the known scent. On the contrary, as we mentioned above, Molet et al. (2009), in B. terrestris, showed that the pre-exposure to an unrewarded odor is sufficient to promote preferential landings on artificial scented flowers. The fact that different bumblebee species were involved and the odors used (anise, peppermint vs. 2-phenylethanol, methyl salicylate: which could differ in their salience) may account for the discrepancies between the studies above mentioned. Additionally, while Molet et al. (2009) tested short-term memory (within an hour), McAulay et al. (2015) evaluated long-term memory (three and 6 days). Such dissimilar time span could trigger neural changes which become consolidated or not according to the presence/absence of association of the scent with a reward.

Finally, an alternative explanation for the improved learning performance of bumblebees preexposed to scented food would involve sensory pseudoconditioning. This phenomenon could be ruled out since the control group (fed with unscented sucrose solution prior to training) did not show such positive effect in the acquisition, suggesting that the improvement found would be the consequence of the previous odor-reward association (Mackintosh, 1994), instead of an alternative effect.

Concerning social learning, in both stingless bees and honey bees, appetitive learning (scent associated with a gustatory reward; Takeda, 1961; Bitterman et al., 1983; Menzel, 1999) via trophallactic food exchanges with successful foragers influences the foraging decisions of individuals naïve to food sources (stingless bees: Jarau, 2009; Mc Cabe and Farina, 2009; Mc Cabe et al., 2015; honey bees: Farina et al., 2005, 2007; Farina and Grüter, 2009; Balbuena et al., 2012a). In contrast, bumblebee foragers do not perform trophallaxis and cannot communicate spatial information about rewarding food sources, but they can provide odor information from rewarding flower species to

#### REFERENCES


their nestmates. In these insects, the crop unloading is done directly into the honeypots by the foraging bee (Dornhaus and Chittka, 2005). These honeypots are the source of the olfactory information stored inside of the colony because a bumblebee probes the nectar contained in them and then goes out to forage (Dornhaus and Chittka, 2005).

Despite the fact that our results do not demonstrate that bumblebees B. atratus are capable of social learning, like numerous other social insects (honey bees: Farina et al., 2005, 2007; Balbuena et al., 2012a,b; bumblebees of other species: Dornhaus and Chittka, 1999; Molet et al., 2009; McAulay et al., 2015), the present study is the first step to understand the mechanisms involved in the recruitment and communication capacity of this particular bumblebees species. Future research may focus on learning associations of scents and food stored in honeypots within the bumblebee nest, where the information transfer takes place, to evaluate its social learning capacity. Such studies on the associative conditioning of floral odors and a sucrose reward could be useful as a tool to influence the foraging behavior of bumblebee workers, opening the possibility to improve the nest management during the pollination services. Furthermore, it is a matter of relevance bearing in mind the fact that B. atratus is increasingly reared as an alternative native species and the potential risk of invasion by exotic bumblebees.

# AUTHOR CONTRIBUTIONS

FP, ME, and WF: conceived and designed the experiments and wrote the article. FP and ME: performed the experiments and analyzed the data. WF: contributed reagents/materials/analysis tools.

## ACKNOWLEDGMENTS

The authors thank Biobest Argentina S.A. for providing the bumblebee colonies during the experimental period. They also thank the CONICET and the University of Buenos Aires for support. This study was partly supported by grants from ANPCYT, University of Buenos Aires, and CONICET to WF.

#### SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpsyg. 2018.00603/full#supplementary-material


foraging decisions. Behav. Ecol. Sociobiol. 66, 445–452. doi: 10.1007/s00265- 011-1290-3


Heinrich, B. (2004). Bumblebee Economics. Cambridge: Harvard University Press.


bees. Entomol. Exper. Appl. 90, 123–129. doi: 10.1046/j.1570-7458.1999.00 430.x



**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2018 Palottini, Estravis Barcala and Farina. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Mind Control: How Parasites Manipulate Cognitive Functions in Their Insect Hosts

Frederic Libersat\*, Maayan Kaiser and Stav Emanuel

Department of Life Sciences and Zlotowski Center for Neurosciences, Ben-Gurion University of the Negev, Beer-Sheva, Israel

Neuro-parasitology is an emerging branch of science that deals with parasites that can control the nervous system of the host. It offers the possibility of discovering how one species (the parasite) modifies a particular neural network, and thus particular behaviors, of another species (the host). Such parasite–host interactions, developed over millions of years of evolution, provide unique tools by which one can determine how neuromodulation up-or-down regulates specific behaviors. In some of the most fascinating manipulations, the parasite taps into the host brain neuronal circuities to manipulate hosts cognitive functions. To name just a few examples, some worms induce crickets and other terrestrial insects to commit suicide in water, enabling the exit of the parasite into an aquatic environment favorable to its reproduction. In another example of behavioral manipulation, ants that consumed the secretions of a caterpillar containing dopamine are less likely to move away from the caterpillar and more likely to be aggressive. This benefits the caterpillar for without its ant bodyguards, it is more likely to be predated upon or attacked by parasitic insects that would lay eggs inside its body. Another example is the parasitic wasp, which induces a guarding behavior in its ladybug host in collaboration with a viral mutualist. To exert long-term behavioral manipulation of the host, parasite must secrete compounds that act through secondary messengers and/or directly on genes often modifying gene expression to produce long-lasting effects.

#### Edited by:

Martin Giurfa, UMR 5169, Centre de Recherches sur la Cognition Animale (CRCA), France

#### Reviewed by:

Hans-Joachim Pflueger, Freie Universität Berlin, Germany Hitoshi Aonuma, Hokkaido University, Japan

> \*Correspondence: Frederic Libersat libersat@bgu.ac.il

#### Specialty section:

This article was submitted to Comparative Psychology, a section of the journal Frontiers in Psychology

Received: 04 March 2018 Accepted: 04 April 2018 Published: 01 May 2018

#### Citation:

Libersat F, Kaiser M and Emanuel S (2018) Mind Control: How Parasites Manipulate Cognitive Functions in Their Insect Hosts. Front. Psychol. 9:572. doi: 10.3389/fpsyg.2018.00572 Keywords: cognition, behavioral manipulation, insects, parasitoids, parasites, hosts, brain

# INTRODUCTION

The ability of parasites to alter the behavior of their hosts has recently generated an unusual interest in both scientists and non-scientists. One reason is that parasites alter the behavior of their host in such a way as to suggest a hijacking of their ability to make decisions. However, how parasites manipulate their hosts is not an esoteric topic, fascinating with its evocation of gruesome zombie movies involving body snatchers. It is rather the understanding of these processes provide fundamental insights into the neurobiology of behavior. Although our understanding of the neural mechanisms of parasitic manipulation is still lacking, there have been some major advances over the past few years. Since most animals are insects, it is not surprising that many case studies of animals that are manipulated by parasites are insects. The diversity of parasites that can manipulate insect behavior ranges from viruses to worms and also includes other insects that have evolved to become parasites (Hughes and Libersat, 2018). In this short review, we will focus on mind

control or the manipulation of cognitive functions in Parasite– Insect associations. We will consider cognition here in a broad sense as the ability of insects to behave not just like reflex machines or automatons (Webb, 2012), but that insects are capable of informed choice-making and goal-directed behavior in a dynamic environment. Recent accumulating evidence demonstrates that insects are more than just automatons and capable of expressing endogenously-created patterns of spontaneous behavior (Perry et al., 2017). For instance, when a single odor is presented to fruit flies in a T-maze at two different concentrations that are easy to tell apart, they make quick decisions and moved to the correct and rewarded end of the chamber. However, when presented with two very near concentrations of the same odor which are difficult to tell apart, the flies take much longer to make a decision leading also to more mistakes. This increase in reaction time when faced with poor quality of sensory information indicates a decisionmaking process in their tiny brains (DasGupta et al., 2014). Furthermore, when fruit flies fly in a white and completely featureless arena, they express endogenously-created patterns of spontaneous behavior (Maye et al., 2007). This suggests a nonrandom endogenous process of behavioral choice, which might imply a precursor motif of "spontaneous" behavior (as opposed to reflexive behavior).

We will first address manipulations that affect an individual host. For the sake of clarity, we have classified these into three general categories: (1) those that affect the compass or navigation of the host that leads to a suicidal behavior. (2) Those which induce the so-called bodyguard behavior. (3) Those that affect the host motivation to move. Then, with some insect species being social and living in colony, we will address manipulations that affect the individual in a social context. Regarding the latter, we will highlight examples of manipulation where the individual, when infected, shows "antisocial" behavior.

#### SUICIDAL BEHAVIOR

Some parasitic fungi and worms manipulate their host's navigational system in most strange ways. Such manipulation ends with the suicide of the host. For example, an ant falling victim to parasitic fungus of the genus Cordyceps is manipulated to produce a behavior that facilitate dispersal of the fungus, thereby optimizing the parasite's chances of reproduction (Hughes, 2015). To this end, Cordyceps fungi produce chemicals that alter the navigational sense of their ant hosts. It begins with the attachment of the spores of the fungus to the cuticle of the ant. The spores then germinate and break into the ant's body by diffusing through the tracheae. Then, fungal filaments called mycelia grow by feeding on the host's organs, avoiding, however, vital ones. The fungus then produces certain, yet unidentified, chemicals that cause the ant to climb to the top of a tree or plant and clamp its mandibles around a leaf or leaf stem to stay in place, a behavior that has never reported for uninfected ants. When the fungus is ready to produce spores, it eventually feeds on the ant's brain and thus kills it. The fruiting bodies of the fungus then sprout out of the cuticle and release capsules filled with spores. The airborne capsules explode on their descent, spreading the spores over the surrounding area to infect other ants and thus start another cycle (Hughes et al., 2011).

Ants can also fall victim to another parasite with a strategy to facilitate the transmission from the intermediate host (the ant) to the final host (a grazing animal). The Lancet liver fluke (Dicrocoelium dendriticum) takes over the ant's (Formica fusca) navigational skills to coerce it into climbing to the tip of a blade of grass (Hohorst and Graefe, 1961). In this position, the ant waits for its deadly fate: being eaten by a grazing animal. The cycle starts with the mature Lancet fluke housing in the liver of the grazing animal and producing eggs which are expelled in the digestive system of the grazer to end up in its feces. Snails get infected by feeding on such droppings. The fluke larvae settle in the snail to be in turn expelled in slime balls. Ants are fond of these slime balls and after a brief sojourn in the ant's gut, the parasites infest the ant's hemolymph and drift inside its body. Remarkably, only one of those parasites migrates alone to the ant's head and settles next to one of the cerebral ganglia, the sub-esophageal ganglion. In this strategic location, it presumably releases some unknown chemicals to control the ant behavior. When evening approaches and the air cools, the infested ant leaves the colony and moves upward to the top of a blade of grass. Once there, it clamps its mandibles onto the top of the blade and stays, waiting to be devoured by some grazer. At the break of day, if the ant life was spared during the night, it returns to the ground and behaves normally. When evening comes again, the fluke takes control again and sends the ant back up the grass for another attempt until a grazing animal wanders by and eats the grass. And so begins a new cycle for the parasite.

Parasites are not necessarily phylogenetically distant from their host. For instance, the crypt gall wasp (Bassettia pallida) parasitizes oaks. It lays an egg in the stem and larva induces the development of a 'crypt' within growing stems. This 'crypt' serves as protection to the larva until it pupates and digs its way out of the stem. This parasitic wasp can be manipulated by another wasp: the parasitoid crypt-keeper wasp (Euderus set) (Weinersmith et al., 2017). When parasitized, adult gall wasps dig an emergence hole in the crypt wall as they do normally, however, instead of emerging through the hole, they plug the hole with their head and die. This benefits the parasite, instead of having to excavate an emergence hole of its own to avoid being trapped, it can use the host's head capsule as an emergence. Dissections of head-plugged crypts reveal larval and pupal stages of the parasitoid residing partly within the crypt and partly within the host's body.

Crickets and other terrestrial insects can fall victim to hairworms, which develop inside their bodies and lead them to commit suicide in water, enabling the exit of the parasite into an aquatic environment favorable to its reproduction (**Figure 1A**). The mechanisms used by hairworms (Paragordius tricuspidatus) to increase the water-seeking behavior of their orthopteran hosts (Nemobius sylvestris) remain a poorly understood aspect of this manipulative process (Ponton et al., 2011). Results of two earlier proteomics studies suggest that phototaxis alterations (i.e., changes in the responses to light stimuli) could be a part of a wider strategy of hairworms for completion of their life

cycles (Biron et al., 2005, 2006). Specifically, parasite-induced positive phototaxis could improve the encounter rate with water (Biron et al., 2006). This assumption was derived from two arguments. Firstly, in the native forest of southern France, water areas such as ponds and rivers are, at night, luminous openings contrasting with the dense surrounding forest. Thus, light could then be a sensory cue that leads infected arthropods to an aquatic environment (Henze and Labhart, 2007). Secondly, besides this ecological reasoning, proteomics data reveal a differential expression of protein families that may be functional components of the visual cycle in the central nervous system of crickets harboring hairworms (Biron et al., 2006).

# OFFSPRING CARE

Although solitary insects are not known to provide care and safety to their offspring, one of the most fascinating behavioral manipulations of parasites is to coerce a host to care for the parasite's offspring. This manipulation is known in insect parasitoids and consists in coercing the host in providing protection to the parasite's offspring from predators (the socalled "bodyguard manipulation"). Protection of this form has been reported for various caterpillar-wasp associations. First, the wasp (A member of the Glyptapanteles species) stings and injects her eggs into the caterpillar (Thyrinteina leucocerae) (Grosman et al., 2008). The caterpillar quickly recovers from the attack and resumes feeding. The wasp larvae mature by feeding on the host, and after 2 weeks, up to 80 fully grown larvae emerge from the host prior to pupation. One or two larvae remain within the caterpillar while their siblings perforate the caterpillar body and begin to pupate. After emergence of the larval wasps to pupate, the remaining larvae take control of the caterpillar behavior by an unknown mechanism, causing the host to snap its upper body back and forth violently, deterring predators and protecting their pupating siblings (**Figure 1B**). Un-parasitized caterpillars do not show this behavior. This bodyguard behavior results in a reduction in mortality of the parasitic wasp offspring. Interestingly, this aggressive behavior of the caterpillar toward intruders must be a component of the host's behavioral repertoire that is usurped by the parasitoid to fulfill another purpose beneficial to the wasp.

Another species of wasp manipulates its host even after leaving the host's body. In the exquisite manipulation, the wasp (Dinocampus coccinellae) inserts one egg only into a ladybug (Coleomegilla maculata) and after emergence of the larva, the ladybug guards the cocoon (Maure et al., 2013). Initially, the single wasp larva develops inside the body of its host, but after about 20 days, it emerges from the ladybug's body and spins a cocoon between its legs. Once the wasp larva has emerged, the ladybug remains alive on top of the cocoon (**Figure 1C**), twitching its body to keep the single wasp pupa safe from potential predators such as lacewings (Dheilly et al., 2015). The survival rate of cocoons protected by living ladybugs from a lacewing predator (another insect) is roughly 65%. If cocoons are left unprotected or attached to dead ladybugs, none or at best 15% survive. Thus, the ladybug, as a bodyguard of the wasp offspring is similar in function to that of the preceding example. Given that the wasp pupa is outside of the ladybug body, and no siblings remain inside the ladybug body, how does this manipulation occur? It appears the wasp injects together with an egg, a virus. The larval-stage parasite contains the virus, and just before the larva exits the host to pupate (and benefits from the bodyguard behavior), it experiences a massive increase in viral replication which are transmitted to the ladybug. The virus replication in the host's nervous tissue induces a severe neuropathy and antiviral immune response that correlates with the symptoms characterizing the motor twitches that serve to protect the pupa (Dheilly et al., 2015). Hence, the virus is apparently responsible for the behavioral change because of its invasion of the ladybug's brain and the virus clearance correlates with behavioral recovery of the host.

On the surface, the interactions between the caterpillar (Narathura japonica) and the ants (Pristomyrmex punctatus) looks like an evolved mutualism (an association between two organisms of different species that beneficial to both organisms). But with a closer look, the caterpillar, which is tended by ants, provides the ants with a secreted substance (sugar-rich secretions) which makes the attendant ants more aggressive. When more aggressive, the ants are less likely to move away from the caterpillar, thereby reducing the chances that the caterpillar would be targeted by predators (Hojo et al., 2015). Although the caterpillar does not invade the ant's body, the researchers found elevated levels of Dopamine in the ant's nervous system.

# SPONTANEITY

The neuronal underpinnings responsible for behavioral spontaneity in insects remain elusive. In our laboratory, we are exploring a unique and naturally-occurring phenomenon in which one insect uses neurotoxins to apparently "hijack" the decision-making ability of another. This phenomenon, a result

of millions of years of co-evolution between a cockroach and its wasp parasitoid, offers a unique opportunity to study the roots and mechanisms of spontaneous behavior in non-human organisms. So far, our investigations point to one possible neuronal substrate involved in the regulation of spontaneous behavior in insects.

The cockroach central nervous system comprises two cerebral ganglia in the head, the supraesophageal ganglion ('brain') and the subesophageal ganglion (SEG). The cerebral ganglia have been implicated in controlling expression of locomotor patterns that are generated in the thoracic ganglia (Kien and Altman, 1992; Schaefer and Ritzmann, 2001). The thoracic ganglia house networks of inter- and motoneurons, which, among other functions, generate the motor patterns for flight and walking. In the brain, numerous investigations suggest that a central structure called the central complex (CX), which is involved in sensory integration and pre-motor processing, is also involved in ongoing regulation of locomotion. For instance, in cockroaches, some CX units show increased firing rates preceding initiation of locomotion and stimulation of the CX promotes walking, indicating that the CX is predominantly permissive for walking (Bender et al., 2010). The Jewel Wasp (Ampulex compressa) stings cockroaches (Periplaneta americana) (**Figure 1D**) and injects venom into the SEG and in and around the CX in the brain (Haspel et al., 2003). The venom induces a longterm hypokinetic state characterized by the inability of the stung cockroach to initiate walking. Other behaviors such as righting, flying, or grooming are not affected. Although stung cockroaches seldom express spontaneous or evoked walking under natural conditions, immersing them in water is stressful enough to induce spontaneous coordinated walking similar to that observed in un-stung cockroaches. However, stung cockroaches maintain swimming for much shorter durations than un-stung cockroaches, as if they 'despair' faster (Gal and Libersat, 2008). This and other examples suggest that the venom selectively attenuates the ongoing 'drive' of cockroaches to produce walking-related behaviors, rather than their mechanical ability to do so. Our recent data indicate that behavioral manipulation of cockroaches by the jewel wasp is achieved by venom-induced inhibition of neuronal activity in the CX and SEG. Our results show that focal injection of procaine or venom into the CX is sufficient to induce a decrease in spontaneous walking indicating that the CX is necessary for the initiation of spontaneous walking. Furthermore, venom injection to either the SEG or the CX of the brain is, by itself, sufficient to decrease walking initiation (Gal and Libersat, 2010; Kaiser and Libersat, 2015). Hence, our investigation of the neuronal basis of such parasite-induced alterations of host behavior suggests that the parasite has evolved ways to tap on the host's brain circuitry responsible for behavioral spontaneity.

#### SOCIALITY

The organization of insect sociality implies cooperative care of offspring and a division of labor into different castes each with a specific task for the benefit of the society (Michener, 1969). This complex organization can be penetrated by specialist "social parasites" (Barbero et al., 2009). One such parasite is the caterpillar (Maculinea rebeli) which mimics the ants (Myrmica schencki) surface chemistry and the sounds they use to communicate, allowing it to penetrate the ant colonies undetected and enjoy the treats of their queen larvae (Akino et al., 1999; Thomas and Settele, 2004). Ironically, those social parasites are the victims of a parasitoid wasp (Ichneumon eumerus) which deposits its eggs into the caterpillar. The wasp's offsprings emerge later as adults from the caterpillar cocoon. The wasp seeks the caterpillar host by first detecting the ant colonies. The body surface chemicals expressed by the wasp induce aggression in ants, leading to in-fighting between the ants. This distraction permits the wasp to penetrate the nest and attack the caterpillar host.

In fire ant parasitic flies (Pseudacteon tricuspis), the female will strike an ant and inject an egg into the ant's (Solenopsis invicta) body. After the larva hatches, it moves into the ant's head and feeds mostly on hemolymph (the equivalent of blood in insect) until just prior to pupation. The larva then consumes the contents of the ant's head, upon which the head usually falls free of the body. The adult fly will emerge from the ant's head 2–6 weeks after pupation. Unlike un-parasitized ants which die inside the nest, those parasitized by the fly larvae leave the safety of the nest shortly before their decapitation. Yet, when parasitized ants leave their nest prior to decapitation, their behavior is indistinguishable from un-parasitized ants. The host's brain is evidently still intact when the ants leave the colony as it is last consumed by the parasitoid (Henne and Johnson, 2007).

From ants to honeybees; Microsporidia (Nosema ceranae), a unicellular parasite, infection in honey bees (Apis mellifera) affects a range of individual and social behaviors in young adult bees (Lecocq et al., 2016). In social bees, age polyethism refers to the functional specialization of different members of a colony based on age. Infection of bees by the parasite significantly accelerates age polyethism causing them to exhibit behaviors typical of older bees. Infected bees also have significantly increased walking rates and higher rates of trophallaxis (food exchange) (Lecocq et al., 2016).

Switching from social bees to social wasps, a fly-like larva (Xenos vesparum) waits for a wasp (Polistes dominula) to land nearby and strikes, penetrating the wasp cuticle to dwell into its abdomen and feeds on its blood (Beani, 2006). Paper wasps are eusocial animals, the highest organization of sociality in animals. When infected with the fly parasite, the normally social wasp starts withdrawing from its colony showing some erratic behavior for no apparent reason other than the presence of the parasite inside it body, messing up with its brain (Hughes et al., 2004). Eusocial colonies include two or more overlapping generations, show cooperative brood care and are divided into reproductive and non-reproductive castes. Individuals of at least one caste usually lose the ability to perform at least one behavior characteristic of individuals in another caste (Michener, 1969). Paper wasp colonies are founded in the spring by one or several

females gynes (non-working pre-overwintering queens), who build the nest and rear a first generation of female workers. The founding female will become the primary reproductive colony queen, while the workers perform tasks such as nest building and brood care. Later in the colony cycle, larvae are reared by workers and emerge as males or female gynes. Those gynes leave the colony in the fall to form aggregations outside the colony with other gynes, where they spend the winter until they scatter to find new colonies in the spring. Female wasps infested by the flylike larva undergo dramatic behavioral changes. Although those females should be workers they behave as typical gynes: they show nest desertion and formation of pre-overwintering aggregations. This behavior is beneficial for the mating and distribution of the parasite (Hughes et al., 2004). In early summer, the infected wasp just leaves its colony behind on a journey to a meeting place with other infected wasps. Male and female parasites can then mate. Whereas wasps infected by male flies die, those infected by females remain alive and under the control of their parasites. They begin to act like wasp zombie queens feeding and growing until they go back to their or other colonies loaded with fly larvae to infect their sister wasps. RNA-sequencing data used to characterize patterns of brain gene expression in infected and non-infected females shows that infected females show gyne brain expression patterns. These data suggest that the parasitoid affects its host by exploiting phenotypic plasticity related to social caste, thus shifting naturally occurring social behavior in a way that is beneficial to the parasitoid (Geffre et al., 2017).

# CONCLUSION

For comparison, the best-studied example of parasitic manipulation of cognitive function in mammals is the case of Toxoplasmosis, an illness caused by the protozoan parasite Toxoplasma gondii. It infects rodents such as mice and rats (the intermediate host) to complete its life cycle in a cat (the final host). The parasite infects the brain forming cysts that produce an enzyme called tyrosine hydroxylase, the limiting enzyme to make dopamine. The most conspicuous behavioral modification in the rat is a switch from avoidance to attraction to cat urine (Berdoy et al., 2000). In doing so, the parasite facilitates its own transmission from the intermediate host to the final host. Such a specific behavioral changes suggests that the parasite finely modify the brain neurochemistry of its intermediate host to facilitate predation, leaving other behavioral traits untouched. This has led to the hypothesis that the host brain is overflown with excess dopamine produced by the parasite, hence, making dopamine the primary suspect in this manipulation. Recently, the parasite genes that encode tyrosine hydroxylase have been identified. By generating a tyrosine hydroxylase mutant parasitic strain of toxoplasma, it was possible to test directly the involvement of dopamine in the manipulation process (Afonso et al., 2017). The authors reported that both mice infected with wild type or mutant (enzyme deficient) strains showed both changes in exploration/risk behavior.

Although humans are dead-end host for the parasite, humans can be infected and some scientists have suggested that T. gondii infection can alter human behavior. Because the parasite infects the brain, it is suspected of making people more reckless, even being liable for certain cases of schizophrenia (Fuglewicz et al., 2017). However, such a hypothesis is still highly controversial and will require more investigations. Today, modern humans are not suitable intermediate hosts because big cats no longer prey upon them. Hence, behavioral modifications in humans could represent a residual manipulation that evolved in appropriate intermediate hosts. An alternative hypothesis, however, states that these changes result from parasite manipulative abilities that evolved when human ancestors were still under significant feline predation. In order to understand the origin of such behavioral change in humans, a recent study tested chimpanzees, which are still preyed upon in their natural environment by leopards. The behavioral test centered on olfactory cues showed that, whereas uninfected individuals avoided leopard urine, parasitized individuals lost this aversion (Poirotte et al., 2016). In the frame of the human evolution, hominids have long coexisted with large carnivores and were considered as good as a meal as our distant and extinct cousins. Hence, when big cats were chasing our ancestors, T. gondii manipulative skills could have evolved because early hominids were suitable intermediate hosts.

Beyond the awe with which we observe the amazing parasitic manipulations described in this review, there is a need to investigate the proximate mechanisms of such behavioral manipulations. Although our understanding of the neural mechanisms of parasitic manipulation is still in its infancy, there have been some major progresses mostly due to advances in molecular biology, biochemistry and biological engineering. Even with tiny quantities of the parasite's secretome (secretions produced by the parasite that may be involved in the host nervous system manipulation), we can use metabolomic, proteomic, and transcriptomic approaches to characterize the library of the secretome components. However, deciphering the composition of the parasite secretome is only the first necessary step. The next and more challenging step is to determine a causal relationship between individual secretome components and their contribution to the observed behavioral manipulation of the host. One promising avenue to address this challenge relies on the recent availability of gene editing tools such as RNA interference (a method of silencing gene product for editing the secretome content) and CRISPR Cas-9 (a method for editing parts of the genome in the parasite). By combining these tools, we are getting closer to unravel the molecular mechanisms of these extraordinary behavioral manipulations.

# AUTHOR CONTRIBUTIONS

All authors listed have made a substantial, direct and intellectual contribution to the work, and approved it for publication.

# REFERENCES

fpsyg-09-00572 April 18, 2018 Time: 17:34 # 6


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2018 Libersat, Kaiser and Emanuel. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# How to Navigate in Different Environments and Situations: Lessons From Ants

Cody A. Freas1,2 and Patrick Schultheiss<sup>3</sup> \*

<sup>1</sup> Department of Biological Sciences, Macquarie University, Sydney, NSW, Australia, <sup>2</sup> Department of Psychology, University of Alberta, Edmonton, AB, Canada, <sup>3</sup> Research Center on Animal Cognition, Center for Integrative Biology, French National Center for Scientific Research, Toulouse University, Toulouse, France

Ants are a globally distributed insect family whose members have adapted to live in a wide range of different environments and ecological niches. Foraging ants everywhere face the recurring challenge of navigating to find food and to bring it back to the nest. More than a century of research has led to the identification of some key navigational strategies, such as compass navigation, path integration, and route following. Ants have been shown to rely on visual, olfactory, and idiothetic cues for navigational guidance. Here, we summarize recent behavioral work, focusing on how these cues are learned and stored as well as how different navigational cues are integrated, often between strategies and even across sensory modalities. Information can also be communicated between different navigational routines. In this way, a shared toolkit of fundamental navigational strategies can lead to substantial flexibility in behavioral outcomes. This allows individual ants to tune their behavioral repertoire to different tasks (e.g., foraging and homing), lifestyles (e.g., diurnal and nocturnal), or environments, depending on the availability and reliability of different guidance cues. We also review recent anatomical and physiological studies in ants and other insects that have started to reveal neural correlates for specific navigational strategies, and which may provide the beginnings of a truly mechanistic understanding of navigation behavior.

Keywords: navigation, ants, path integration, sky compass, terrestrial panorama, landmarks, central complex, mushroom bodies

# INTRODUCTION

Successful navigation requires animals to acquire and apply environmental cues indicating the direction and distance of goal locations. Foraging ants are excellent navigators despite their low visual acuity (Schwarz et al., 2011a; Graham and Philippides, 2017), and their varying navigational strategies have been widely studied (Zeil, 2012; Collett et al., 2013; Cheng et al., 2014; Wehner et al., 2016). These strategies include landmark-based guidance using the panorama (Wehner, 2003; Cheng et al., 2009) and path integration (Collett and Collett, 2000; Wehner, 2003, 2008) with systematic search functioning as a back-up (Schultheiss and Cheng, 2011; Schultheiss et al., 2015). Many of the elements of this navigation toolkit are shared with other social hymenopterans such as bees and wasps, which have been studied in great detail (Collett and Collett, 2002; Cheng, 2006; Zeil, 2012).

Path integration allows the navigator to update its current position relative to the nest by coupling a distance estimate, pedometer-based in ants, with directional estimates from the celestial compass. This coupling results in a working memory-based vector which points the navigator home. As the ant returns to the nest, it runs off this vector which resets once the ant re-enters

#### Edited by:

Jeffrey A. Riffell, University of Washington, United States

#### Reviewed by:

Zhanna Reznikova, Institute of Systematics and Ecology of Animals (RAS), Russia Mathieu Lihoreau, Centre National de la Recherche Scientifique (CNRS), France

#### \*Correspondence:

Patrick Schultheiss ppschultheiss@gmail.com

#### Specialty section:

This article was submitted to Comparative Psychology, a section of the journal Frontiers in Psychology

Received: 01 March 2018 Accepted: 09 May 2018 Published: 29 May 2018

#### Citation:

Freas CA and Schultheiss P (2018) How to Navigate in Different Environments and Situations: Lessons From Ants. Front. Psychol. 9:841. doi: 10.3389/fpsyg.2018.00841

**56**

the nest, yet there is also evidence that ants retain long-term memories of previous vectors (Ziegler and Wehner, 1997).

Landmark use in ants involves the learning of cues present in the panorama (Wehner et al., 1996; Graham and Cheng, 2009; Wystrach et al., 2011). These stored panorama cues are subsequently compared to current views when navigating (Collett et al., 2006; Cheng et al., 2009; Zeil et al., 2014a). How ants acquire, retain, and use both the panorama and other learned cues while foraging continues to be a topic of interest (Knaden and Graham, 2016).

Within this review, we discuss three main avenues of current research in ant navigation. We first summarize the ability of these navigators to learn and retain navigational information from their environment, focusing on panorama cues. Next, we explore current work on how foragers integrate different cues during navigation and how this integration affects cue choice. Finally, we outline the current understanding of the neural architecture underlying these abilities.

# LEARNING AND MEMORY

#### Learning Walks

Using panorama-based navigation first requires the acquisition of cues around the nest through multiple pre-foraging learning walks (Nicholson et al., 1999; Baddeley et al., 2011; Zeil et al., 2014a). During these walks, foragers meander near the nest entrance, likely learning the panorama makeup around the nest (Wehner et al., 2004). Recent work continues to expand our understanding of these walks, focusing on the genus Cataglyphis. Cataglyphis fortis, a desert species living with few panorama cues, exhibits learning walks that first occur within a few centimeters of the nest entrance, with each subsequent walk becoming wider. These ants typically complete 3–7 walks before the onset of foraging and show clear evidence of improved learning of panorama cues after these learning walks (Fleischmann et al., 2016, 2018). Learning walks appear to be mediated by the environment, as species inhabiting landmark-rich environments (Cataglyphis aenescens and Cataglyphis noda) will occasionally 'pirouette' and turn back to the nest, likely learning panorama cues (Fleischmann et al., 2017). These pirouettes are observed in some barren-habitat species (Ocymyrmex robustior, Müller and Wehner, 2010) but not in the widely studied C. fortis. Conversely, C. fortis foragers walk in loops without stopping, even when landmark cues are artificially present (Fleischmann et al., 2017). Interestingly, the absence of pirouetting does not prevent this species from learning these cues during these walks (Fleischmann et al., 2016). Species-specific differences in terrestrial cue learning during these walks, as well as those of species outside of Cataglyphis and Ocymyrmex remain largely unstudied and a ripe topic of future research.

#### Use of the Panorama

During learned panorama-based navigation, the specific cues in use remain highly debated, as what visual cues and aspects of the panorama are used for directional guidance remains uncertain. Most prevalent models involve view-based matching, where foragers compare stored views with their current view to direct them to goals (Zeil et al., 2003; Möller, 2012). Research has also focused on the use of the skyline pattern/height as navigational cues (Graham and Cheng, 2009). The desert ant Melophorus bagoti has been shown to have the ability to use skyline cues through the presence of the UV contrast between the sky and ground to orient successfully as well as retaining skyline cues over long periods (Schultheiss et al., 2016; Freas et al., 2017c). Another view-based strategy of current interest consists of ants' use of the fractional position of mass of the visual scene when comparing stored views and current views (Lent et al., 2013). Here, ants acquire the fraction of the terrestrial scene to the left and right while facing the goal, comparing these stored views to their current view while navigating. When only a single terrestrial object is visible, foragers appear to learn the position of the object's center of mass within stored views and attempt to place this center of mass in the same retinal position when navigating (Buehlmann et al., 2016; Woodgate et al., 2016).

#### Responding to Panorama Changes

Given that natural cues do not remain constant, ants will occasionally experience changes in the panorama either at their nest or along known routes. Consequently, ants need to be able to respond to these changes while navigating. The nocturnal bull ant Myrmecia pyriformis is highly sensitive to panorama changes. When several trees were removed, resulting in small changes to the nest panorama, foragers showed major disruptions in their navigational efficiency, walking slower and less directed. Furthermore, these behavioral changes persisted over multiple nights before returning to pre-change levels, suggesting a period of relearning the new panorama (Narendra and Ramirez-Esquivel, 2017). Yet there appears to be a range of flexibility across species, as recent work in M. bagoti suggests foragers learn new panoramas after only one exposure (Freas and Cheng, 2017, 2018a) and can successfully orient to both new and old panoramas for multiple days after a change occurs (Freas et al., 2017c).

Navigating ants also exhibit interesting behaviors when panorama discrepancies occur due to their position in threedimensional space. When foraging on non-level surfaces, M. pyriformis will attempt to roll their head, keeping it close to the horizontal plane. This behavior is believed to reduce visual noise when comparing memorized views with current views, as similarity declines as the view is rotated (Raderschall et al., 2016). An extreme form of this behavior appears to be present while foragers' bodies are positioned vertically on trees. Myrmecia midas foragers perform scans where they roll or pitch their head toward the horizontal plane while the body remains vertical. This behavior may be an attempt to align their current views with memorized views on the ground (Freas et al., 2018).

#### Learning Other Cue Sets

While panorama cues are currently the most widely studied form of learning, new research suggests ant navigators can learn a variety of cue sets and associate them with the nest. Cataglyphis foragers can also learn associations using local olfactory, magnetic, and vibrational cues. Cataglyphis noda will

search at locations with locally distinct magnetic, vibrational, and olfactory signatures when these had previously been paired with the nest entrance (Buehlmann et al., 2012a). Additionally, olfactory cues can be learned in association with locations beyond the nest site as part of the foraging route. Cataglyphis fortis foragers have been shown to learn odor landmarks along their foraging route after training (Buehlmann et al., 2015). Recent work in the Cataglyphis genus is unveiling that the role of olfactory cues has been understudied as a navigational cue set for both nest and food locations (Buehlmann et al., 2012a,b, 2013, 2014, 2015).

#### INTEGRATION OF NAVIGATIONAL INFORMATION

#### Directional Cue Integration

On featureless saltpans, without visual guidance cues, C. fortis foragers use path integration not only to return home, but also to return to previously visited goals. To achieve this, they compare a memorized vector that would lead directly to the goal with the current state of the path integrator (essentially performing vector summation) and deriving a direction in which to move (Collett et al., 1999). This combination of two vector memories, one longterm and one short-term, thus enables them to navigate to a goal. When the previous inbound memory and the current outbound route mismatch consistently, this system adapts by calibrating vectors at recognized sites. Recent experiments on M. bagoti revealed that the homeward vector memory recalibrates rapidly, with the inbound vector dominating when the mismatch is small (45◦ ). As the mismatch increases, calibration toward the inbound vector decreases with ants showing no calibration at the maximal mismatch (180◦ ), where the current vector dominates (Freas and Cheng, 2018b).

Such integration of different directional dictates has been found repeatedly in ant navigation studies, and has attracted particular interest as it can show which navigational processes are engaged simultaneously, and how they might be organized in the insect brain (Wehner et al., 2016). Under natural foraging conditions, ants often have multiple sets of guidance cues available simultaneously, and information sharing and integration can occur between different navigational systems. The desert ant Cataglyphis velox, for example, navigates home using path integration and memorized terrestrial visual cues. Normally these two systems provide redundant directional information but, when put into conflict, these ants choose intermediate directions. However, during path integration the variance of the directional estimate decreases with vector length, so that after long runs the directional dictate from path integration can be more certain than that from visual memory. The merging of directional information from the two systems has been shown to happen in an optimally weighted manner, taking this relative certainty into account (Wystrach et al., 2015).

#### Cue Integration During Learning

The role of learning terrestrial visual cues in such conflict situations has also been explored in more detail in M. bagoti (Freas and Cheng, 2017). Foragers restricted to the nest site could not extrapolate visual panorama information to a local (8 m) site. While one exposure to this new panorama was sufficient for successful homing, it did not override a conflicting vector direction. Repeated exposure to the new panorama increased the weighting of these cues, eventually overriding vector information. Interestingly, this pattern of cue choice appears to be dynamic, as terrestrial cues were increasingly discounted with time since last exposure, consistent with the temporal weighting rule (Devenport and Devenport, 1994). View sequence may also be important during landmark learning, as foragers encountering only the inbound view sequence show weaker panorama learning and a higher propensity to switch to vector cues compared to foragers exposed to outbound views (Freas and Cheng, 2018a). Highly visually experienced foragers are not only better at using the panorama for homing, but also better at recognizing changes. Training ants to visit a feeder, Schwarz et al. (2017b) compared visually experienced ants with naïve ants visiting the feeder for the first time. When released in unfamiliar surroundings, naïve ants ran off a longer portion of the path integration vector, while experienced ants broke off their directed travel route earlier. Being familiar with the nest's surroundings, they could more readily realize that the view was unfamiliar and engage in searching behavior. Buehlmann et al. (2018) investigated the walking speed of C. fortis on homeward runs, finding that they slow down when approaching the nest; they are also more alert to visual changes closer to the nest. Interestingly, the relevant cue for these behavioral changes is the completed proportion of the homing vector, suggesting that path integration modulates speed in a way that facilitates the use or learning of visual cues at important locations.

Myrmecia midas also orients by both celestial and terrestrial visual cues on outbound trips, and manipulating the direction of polarized overhead light leads to compromises between the directional dictates of celestial and terrestrial cues throughout the outbound journey (Freas et al., 2017b). When orienting on inbound journeys however, they appear to use celestial information only when the accumulated homing vector is large (Freas et al., 2017a). Accordingly, the weighting of celestial cues also scales with vector length (Freas et al., 2017b). Weighted integration of visual cues can therefore be context-dependent. For C. fortis ants on salt-pans, the CO2 plume emitted by the nest can be an important guidance cue, however, ants will only follow this cue when their homing vector is close to zero (Buehlmann et al., 2012b). This might prevent foragers from mistakenly entering conspecific nests, as CO2-plumes are not nest-specific. Such vector-dependence does not apply to food odors, as ants will respond to these regardless of the state of the path integrator (Buehlmann et al., 2013). These findings illustrate that cue integration can function across sensory modalities, in a context-dependent manner.

#### Communication Between Navigational Strategies

Information can also be communicated between two navigation systems. Intrigued by the fact that ants can maintain straight

compass directions even when walking backward (dragging large food) recent studies on Cataglyphis have shown that these backward journeys are frequently interrupted. The ants briefly drop the food and perform small search loops (Pfeffer and Wittlinger, 2016b) or short forward 'peeks.' These peeks allow them to use the visual panorama to update their compass heading and transfer this heading to celestial cues (Schwarz et al., 2017a). As the celestial compass can function independently of body orientation, this is then used during backward walking, when the panorama is misaligned (Collett et al., 2017). In other cases, information transfer between systems may not always occur, even when these systems naturally provide redundant information. One such case is odometry, in which C. fortis measures the distance traveled by both a stride integrator (Wittlinger et al., 2006) and ventral optic flow (Ronacher and Wehner, 1995). Studying ants that were being carried between sub-colonies, Pfeffer and Wittlinger (2016a) showed that the odometric estimate from optic flow alone was sufficient for

subsequent homeward navigation with intact eyes. Ants that were carried the same way, but then had the ventral eye regions covered could not navigate home although their stride integrator was fully functional, showing that odometric information was not communicated between the two systems. Similarly, M. bagoti has two parallel systems for perceiving celestial compass cues: through the dorsal rim area of their complex eyes, and through the ocelli on top of their heads. However, after a dog-legged outbound route, only compass information from the eyes is available for path integration, while ocelli information can only be used for reversing the last leg of travel (Schwarz et al., 2011b).

#### NEURAL MECHANISMS

To fully comprehend such multi-facetted and flexible navigation behavior on a mechanistic level requires detailed knowledge of

from Grob et al. (2017).

(AOT, shown in blue) leads from the lamina (LA) in the OL to the CX, via the anterior optic tubercle (AOTU) and the lateral complex (LX); scale bar is 200 µm. Adapted

the underlying neuroanatomy and physiology. Insects provide the distinct advantage that, though capable of sophisticated behaviors, their central nervous system comprises relatively low neuron numbers (about 1 million in honeybees; Witthöft, 1967), and an understanding should be feasible. Much neurobiological work has focused on the fruit fly Drosophila and the honeybee Apis mellifera, with considerably less work on ants. Nevertheless, the brains of ants share all the key features with other insects, and with bees in particular (Gronenberg and López-Riquelme, 2004; Bressan et al., 2015).

An overview of an ant brain is shown in **Figure 1**. Visual information enters through the optic lobes, while the antennal lobes process olfactory input. The mushroom bodies (MBs) are centers for sensory integration, learning, and memory (Menzel, 2014). The central complex (CX) is involved in memory, visual processing, and sensorimotor processing (Pfeiffer and Homberg, 2014). The neural basis of the sky compass, using polarized light, is currently best understood (Heinze, 2017). Behavioral and physiological findings have revealed that ants perceive the angle of light polarization (POL) through specialized UVphotoreceptors at the dorsal part of the compound eyes (Labhart and Meyer, 1999; Zeil et al., 2014b). A putative neural skycompass pathway, the anterior optic tract, has been identified (Schmitt et al., 2016), transmitting POL information from the optic lobes to the CX (**Figure 1**). In locusts and Megalopta bees, POL angles are anatomically represented in a systematic manner in a subcompartment of the CX, the protocerebral bridge (PB) (Heinze and Homberg, 2007; Stone et al., 2017). In this way, the PB can encode the animal's global heading, as the direction of POL angles depends on the azimuthal position of the sun. In ants, the neuroanatomy of the CX and its subcompartments is nearly identical and likely functions similarly (Grob et al., 2017).

Recent work in Drosophila has shown how the connectivity of the PB and the lower division of the central body (CBL; another CX subcompartment) together form a ring-attractor network, which is able to track changes in heading and update the neural representation accordingly (Seelig and Jayaraman, 2015). Since the CBL also integrates information from POL neurons and speed neurons (Stone et al., 2017), the CX has been successfully modeled as a path integrator (Goldschmidt et al., 2017; Stone et al., 2017). In ants, it remains unclear how speed might be neurally encoded.

The neural mechanisms of other visual navigation strategies, which rely on long-term memories of landscape features, are less well-understood in ants or other insects. It is clear that the MBs play a significant part in visual processing and memory formation (Menzel, 2014), although the CX can be involved in some of these tasks (Drosophila: Neuser et al., 2008; ants: Grob et al., 2017). The prominent anterior superior optic tract connects the optic lobes with visual subregions of the MBs (Gronenberg, 2001; **Figure 1**). There is good evidence in ants that these regions are involved in visual memory as they undergo considerable neuroanatomical changes after light exposure (Stieb et al., 2010, 2012). The MBs also contain olfactory subregions that receive neural input from the antennal lobes (Gronenberg and López-Riquelme, 2004). In ants, these subregions go through significant structural changes during the formation of olfactory long-term memories (Falibene et al., 2015) and in bees (Apis), the role of MBs in olfactory learning and memory is clearly established (Hourcade et al., 2010). The neural connectivity within Drosophila MBs is in fact so well-understood that it has inspired convincing models of their involvement in olfactory learning (Aso et al., 2014); these have since been adapted to model how image-based memories could be stored (Ardin et al., 2016; see also Webb and Wystrach, 2016). It is not yet known how stored visual information might be compared with currently perceived views, or how MB output signals may be converted into motor commands, as prominent neural connections to the CX have not been identified.

To advance our understanding of ant navigation neurobiology in the near future, it remains essential to further elucidate the main circuitry in the ant brain. Neural connections, predicted by our knowledge in related insects and computational models, need to be investigated and verified. Precise neurophysiology on living ants continues to be a key challenge, especially in ecologically relevant contexts. Major advances in Drosophila neurobiology have been achieved through neural manipulations on tethered animals, and with recent developments of advanced trackball setups for walking hymenopterans (ants: Dahmen et al., 2017; bees: Schultheiss et al., 2017), such avenues may now be open for ants as well.

#### CONCLUSION

Foraging ants have been key to the study of navigational strategies such as path integration, panorama-based guidance, and the use of a bevy of olfactory, visual, and idiothetic cue sets. This review has focused on three avenues representing the current state of work across multiple species, the learning and storing of navigational cues, the integration of multiple information streams while navigating, and the neural and anatomical structures underlying these strategies. Together, these studies provide the base for forming a mechanistic framework for navigational decision making and behavior.

#### AUTHOR CONTRIBUTIONS

PS conceived the study. CF and PS wrote and revised the manuscript.

# FUNDING

The authors did not receive any funding for this study. The publication fees were covered by the Australian Research Council (Grant No. DP150101172) and the Human Frontier Science Program (Grant No. RGP0022/2014).

# ACKNOWLEDGMENTS

We thank Ken Cheng for comments on a previous version of this manuscript.

# REFERENCES

fpsyg-09-00841 May 25, 2018 Time: 17:50 # 6



**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2018 Freas and Schultheiss. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Cognitive Aspects of Comb-Building in the Honeybee?

Vincent Gallo<sup>1</sup> and Lars Chittka1,2 \*

<sup>1</sup> Department of Psychology, School of Biological and Chemical Sciences, Queen Mary University of London, London, United Kingdom, <sup>2</sup> Wissenschaftskolleg zu Berlin, Institute for Advanced Study, Berlin, Germany

The wax-made comb of the honeybee is a masterpiece of animal architecture. The highly regular, double-sided hexagonal structure is a near-optimal solution to storing food and housing larvae, economizing on building materials and space. Elaborate though they may seem, such animal constructions are often viewed as the result of 'just instinct,' governed by inflexible, pre-programmed, innate behavior routines. An inspection of the literature on honeybee comb construction, however, reveals a different picture. Workers have to learn, at least in part, certain elements of the technique, and there is considerable flexibility in terms of how the shape of the comb and its gradual manufacture is tailored to the circumstances, especially the available space. Moreover, we explore the 2-century old and now largely forgotten work by François Huber, where glass screens were placed between an expanding comb construction and the intended target wall. Bees took corrective action before reaching the glass obstacle, and altered the ongoing construction so as to reach the nearest wooden wall. Though further experiments will be necessary, these results suggest a form of spatial planning skills. We discuss these findings in the context of what is now known about insect cognition, and ask if it is possible that the production of hexagonal wax combs is the result of behavioral heuristics where a complex structure emerges as the result of simple behavioral rules applied by each individual, or whether prospective cognition might be involved.

#### Edited by:

Ken Cheng, Macquarie University, Australia

#### Reviewed by:

Susan Healy, University of St Andrews, United Kingdom Zhanna Reznikova, Institute of Systematics and Ecology of Animals (RAS), Russia

#### \*Correspondence: Lars Chittka

l.chittka@qmul.ac.uk

#### Specialty section:

This article was submitted to Comparative Psychology, a section of the journal Frontiers in Psychology

Received: 27 February 2018 Accepted: 17 May 2018 Published: 05 June 2018

#### Citation:

Gallo V and Chittka L (2018) Cognitive Aspects of Comb-Building in the Honeybee? Front. Psychol. 9:900. doi: 10.3389/fpsyg.2018.00900 Keywords: behavior, cognition, consciousness, planning, prediction, prospective cognition, wax

# INTRODUCTION

It has long been recognized that social insects have rich behavioral repertoires that orchestrate life in the colony, facilitate the elaborate construction of a communal home, secure a steady stream of appropriate food for their offspring, defend the colony and regulate its climate. This behavioral complexity has often been dismissed as 'just instinct.' Yet, recent discoveries in insect learning, memory and cognition have generated a profound change in the perception of the behavioral flexibility of several species. For example, bees learn from past experiences to improve motor skills (Mirwan et al., 2015; Abramson et al., 2016). Such operant learning is distinguished from cognitive operations, where, for example, bees are also able to combine multiple experiences (acquired in separate learning trials) to form simple rules and concepts (Giurfa et al., 2001; Avargues-Weber et al., 2012) and display counting-like abilities (Howard et al., 2018; Skorupski et al., 2018), and ants and bumblebees show simple forms of tool use (Loukola et al., 2017; Maák et al., 2017). Being capable of interval timing, bumblebees can predict future events (Boisvert and Sherry, 2006; Skorupski and Chittka, 2006). There is evidence that insects might at some level predict the

outcomes of their own actions (Webb, 2004; Kim et al., 2015; Mischiati et al., 2015), or perceive a desirable outcome and then to explore possibilities to achieve this goal (Chittka, 2017; Menzel, 2017). In view of this, a re-evaluation of some behavioral routines traditionally thought to be entirely governed by instinct is in order (Bateson and Mameli, 2007). Even where behavior is partially instinctual, there can be multiple interactions with learnt behavior and cognition. Bird nest building, for example, was once thought to be wholly instinct-driven, but it is now apparent that many aspects of it can be experience-dependent (Walsh et al., 2013), and indeed the nesting instinct that requires the manipulation of elongated objects such as twigs in some birds can in turn facilitate cognitive behavior such as flexible tool use (Healy et al., 2008; Breen et al., 2016). In view of this, we here re-examine the learnt, and possibly cognitive, elements of what has been regarded by many as the pinnacle of animal instinctual behavior: the construction of the honeybee wax comb (**Figure 1**). Darwin referred to this as "the most wonderful of all known instincts" (Darwin, 1859, p. 235). Here we review the evidence that elements of comb construction need to be learned, and, exploring largely forgotten literature, how cognitive and planning skills might be involved.

### OPTIMALITY OF THE COMB STRUCTURE

The honeybee comb is, at first sight, a wonder of animal architecture. In all known species of honeybees, the structure

FIGURE 1 | Construction of new comb in the honeybee Apis mellifera. The construction of hexagonal honeycombs requires the coordinated and cooperative activities of many dozens of individuals. Workers manufacture and manipulate wax into a highly regular hexagonal pattern (a mathematically close to perfect solution to honey and brood storage), and in the process have to evaluate the space available and the current state of construction, and process a diversity of communication signals from others, as well as proprioceptive input, for example to align the combs with gravity. These rich instinctual repertoires of many insects have often been thought to come at the expense of learning capacity. However, very few behavioral routines are fully hardwired and even comb construction skills have to be partially learnt by honeybees. Image by Helga Heilmann, with permission.

is a double sided sheet of tessellated hexagonal cells where the base (common to both sides) is formed from three rhombi (**Figure 2**). Obviously, hexagonal cells are more suitable than the round cells used by, e.g., bumblebees, since the latter arrangement wastes a lot of space between cells. Square or triangular cells would have no gaps between cells, but since the larvae to be raised in the cells are neither square nor triangular in crosssection, space would be wasted inside the cells. Thus, hexagonal cells are intuitively suitable, and in fact some species of wasps build them too, albeit of "paper" (chewed wood) rather than wax. But no species of bee except honeybees also builds doublesided hexagonal combs — another notable strategy to save space and material. The bottom of each hexagonal cell has the shape of a pyramid (again a more efficient solution than a square bottom), and the two sides of the comb interface perfectly with one another through these pyramid-shaped bases of the cells. Unlike the combs of some stingless bees, the honeybee comb has to be vertical so that honey can be stored on both sides without dripping out, and the cells of the comb are tipped slightly downward from the opening to the base (**Figure 2**). In cavitynesting species (Apis mellifera and A. cerana), multiple combs are built in parallel, leaving just enough space for workers to move about freely (**Figure 3**). This is despite the fact that cavities in which these species of honeybees nest naturally (e.g., hollow trees) are highly irregular in shape (not like the cuboid boxes beekeepers supply them with). Beyond the intuitive arguments in favor of a double sided hexagonal structure, it has been pointed out that the structure is in fact a mathematically optimal, or close to optimal, solution to economizing on building material while maximizing storage space (Kepler, 1611; Huber, 1814/1926; (Langstroth, 1853). Huber (1814/1926, p. 106) reported that the rhombus angles could beneficially be altered by modifying angles by 10 min. Analysis of the geometry of tessellated polyhedrons (Tóth, 1964) showed that the most economical cell construction (volume per wall area) comprised a hexagonal cell with a base formed from two squares and two hexagons. However, the saving would be less than 0.35%, at the expense of greater complexity of construction. By the use of self-aligning soap bubbles (Weaire and Phelan, 1994) it was shown that at a certain wall thickness, the ideal solution would switch from the optimal arrangement proposed by Tóth to that favored by the bees. We can thus infer that the structure is indeed very close to the theoretical optimum.

# CAN WAX COMB CONSTRUCTION BE EXPLAINED BY SIMPLE ALGORITHMS?

The repetitive structure of the comb seems like the perfect result of some robotic, hard-wired behavior routine — a kind of assembly line job of building the same structure over and over. It is tempting to assume some simple algorithm that might explain the shaping of the comb structure. For example, Pirk et al. (2004) proposed that each cell of the comb was a simple structure constructed from a curved wall, cylindrical tube, without facets or edges. The claim was that the temperature and fluid properties of the wax itself at the elevated temperature, present during comb construction, would, without further intervention by the

bees, reform into the seemingly more complex hexagon by liquid equilibrium. This would be a process in which straight surfaces sometimes form in the same way as adjacent soap bubbles. However, thermal imaging technology (Bauer and Bienefeld, 2013) showed that the wax never achieved a temperature sufficient for reformation to occur. The precise geometry is not formed as a natural consequence of the material and temperature, but rather must be actively constructed by the bees, in the same way as wasps (even individual wasp queens) can fashion a hexagonal comb from plant material (Karsai and Pénzes, 2000). The search for parsimonious explanations in animal behavior, however attractive they may be, can sometimes lead in the wrong direction.

In fact, theoreticians sometimes overlook empirical work at odds with their "simple" explanations. While it is often possible to generate a similar outcome as that found in nature by means of modeling or engineering, such exercises can be reminiscent of inspecting a sophisticated piece of medieval

The sketch shows how normal comb constructions of cavity-nesting honeybees where comb is begun attached to the top surface of the cavity, and then gradually extended downwards. Multiple combs will be grown, each roughly parallel and separated by a gap sufficient for the bees to work both. Note that the first line of cells (the "foundation") is differently shaped to other cells. At the lower end of the construction, partially constructed cells come in a large variety of shapes, and individual workers can in principle continue from any partial construction.

embroidery, taking a photo of it, and saying "There! The photo has the same pattern! This means that we now understand how the embroidery pattern was generated." Clearly, even if the result looks similar, we have not understood the technique by which the original was manufactured. This is a fundamental complication with many modeling approaches that try to explore how complex behaviors or constructions might result from "simple rules," including existing ones for comb construction in paper wasps (Karsai and Pénzes, 1998; see Walsh et al., 2013 for an exploration in bird nest building). Any useful model of comb construction would have to take into account, at the very minimum, how slivers of building material are manipulated by an insect's six legs and its mandibles, using its antennae and other sensors to assess where the construction needs to be amended and how, and processing information from other individuals, to ensure that efforts of multiple individuals complement each other to ensure that comb is built efficiently and ideally free of errors (departures from the ideal structure). It would have to consider the mechanistic or algorithmic process whereby a cell could be built by a programmed sequence of steps to masticate, deposit, press, sculpt, remove and replace material so as to form the faces, edges and thus the elemental cell.

The algorithm could be extended with repetition statements to form a regular sequence of cells against a flat horizontal surface, and further algorithm statements would be required to add the partial cells necessary to build horizontal cell layers against a sloped surface. Yet the task undertaken by combbuilding insects goes further and copes with an irregular surface including fractures, cavities, and protrusions as is evident from their inhabitation of cavities within trees and rocks and by open nesting species of honeybees (such as A. florea or A. dorsata) that build externally on tree branches and/or rock. Not only must the builders overcome irregularities rooted in the shape

of the support or cavity but also to notice and overcome errors introduced by themselves or other bees. This is not to say that elements of the social endeavor of comb construction cannot ultimately be explained by stereotyped behavior routines (and possibly in part relatively simple ones), but a model that does not incorporate these natural challenges of comb construction will oversimplify the problem, and generate an illusion of simplicity where there is none.

#### FLEXIBILITY IN HOW INDIVIDUALS BUILD THE COMB

Understanding the behavioral challenges of comb construction requires observation of individual and collective activities of bees engaging in small scale repetitive tasks, executed by many individuals, which collectively can lead to a multi-purpose structure to the benefit of the colony. The dexterity that is required for a six-legged animal to manufacture a repetitive structure with such regularity and precision is remarkable. In his classic work over two centuries ago, Huber (1814/1926) described the many variations that exist in the comb structure: for example, as bees build their comb in the typical manner from the top and working downwards, the first row of cells differs from subsequent ones since it functions as a foundation. One might suspect that worker bees use their own body as a sort of template to arrive at the correct dimensions of each comb cell — but this is certainly only part of the story, since the width of cells destined for drones is 30% larger (yet they are also built by workers). There are multiple other modifications of wax structure, e.g., for the wholly differently shaped cradles for queens, or the entombments for intruders such as mice that have strayed into a colony and are killed by bees. Huber describes in detail how comb construction is initiated by a single worker on the top of the hive, and how multiple individual workers sequentially contribute to the construction of each cell. He also describes the ability of honeybees to shape flat surfaces and angular connections, observing how bees form the rhombic bases by first sculpting the base from a "block" made from balls of wax, softened by a process of chewing and moistening. The beginning shape is subsequently enlarged by the addition of further balls of wax to form the cell walls and edges. The sculpting process, involving removal of surplus material, was described as being undertaken by a number of individuals, both successively and simultaneously, working on diverse sections. Different workers continue cells where others have left off (and do so correctly no matter the previous state of the cell), and inspect one another's constructions to amend them where necessary. Huber(1814/1926, p. 129) noted several bees working on a small area of comb, one of which placed some wax in a misaligned location. An observant coworker was seen relocating the wax better aligned to the current construction. These examples of adaptive behavior are of a small scale, correcting details of a scale less than that of a cell. Cell-scale adaptation of the construction method was also evident when a mixed species colony (A. mellifera and A. cerana) built comb over foundation ideal for one or other species (Yang et al., 2010). The mismatch between the natural cell size and that suggested by the foundation required adaptive modification of the bees' natural construction habit.

Longer range flexible behavior can be seen where two or more festoons (hanging groups of comb forming bees) commence simultaneous construction of comb which, when enlarged, were sufficiently aligned so as to unite into a single blade. To create the connection between the two constructions, pentagons or heptagons are constructed (Hepburn and Whiffler, 1991). In that case the adaptation extends over several cells as to form a junction between misaligned combs. In another example of the bees' flexibility, a hive of bees once traveled on board the Space Shuttle Challenger, 2 years before its doomed final mission in 1986. The honeybees spent an entire week in zero gravity. Not only did they learn to fly under such conditions, but they built honeycomb with cells of normal dimensions. The only difference (compared to honeycomb built on Earth) was that the cells of honeycomb were not consistently angled downward perhaps unsurprisingly, since there is no obvious 'down' in zero gravity conditions for a honeybee (Vandenberg et al., 1985). But importantly, the geometry of the combs was correct — several combs had the usual straight and flat structure, and were built roughly in parallel, in the complete absence of gravity.

In conclusion, detailed observations of the comb building process reveal that multiple behavioral routines might be at work and are subtly tailored to need. Many of them might still be governed by hard-wired, innate routines, but they seem far from simple, given the versatility and flexibility observed. In what follows, we examine the literature indicating that learning and cognition are also involved.

# POSSIBLE ELEMENTS OF LEARNING IN COMB CONSTRUCTION

In natural bee comb constructions, there are a variety of subtly different ways in which wax comb is structured (especially with respect to how the two sides of comb are interfaced). The way in which young workers build comb is affected by the structure of the comb they were raised in, and were allowed to sample for some time after emergence (von Oelsen and Rademacher, 1979). In a similar vein, Martin Lindauer discovered that, after swarming and relocating to new home, the combs in the new home would typically have the same angle to the Earth's magnetic field as the natal nest, indicating that bees had memorized this angle and then replicated it in the new construction (Seeley et al., 2002). While these observations are indicative of an importance of learning in comb construction, it might also be possible that there are genetic effects that determine comb structure and orientation.

The classic approach to investigate whether behavioral routines present in adults are innate or learnt is to experiment on individuals reared in isolation, under conditions where they have no exposure to the behavior in question, or to the desired outcome of the behavior. For questions of comb construction, such experiments were first performed with orphaned Polistes wasps, and it was observed that the comb geometry in such wasps departed from the usual radial symmetry (Rau, 1929).

von Oelsen and Rademacher (1979) reared honeybee larvae, removed from their natal comb, in circular plastic cells. Such individuals later managed to build hexagonal cells, but with highly variable cell dimensions. In addition, peculiar modifications were apparent in the comb structure beyond that of the single cell. Bees raised without having experienced normal honeycomb built irregular bases and unconventional cell arrangements (rotated or floral configuration) while juveniles that had been allowed access to conventional comb and/or experienced workers built conventional comb. As with many other behaviors, e.g., bird song, innate predispositions only provide a rough template for acceptable behavior in the adult the details need to be learned (Thorpe, 1973; Mets and Brainard, 2018).

Learning can also be apparent in insects' ability to repair experimentally damaged comb. Working with Polistes wasps, Downing and Jeanne (1990) observed that when holes were made in the existing comb structure, the time to repair them decreased with repeated exposure, and individual wasps improved their repair technique with experience. For an example of repairing accidental damage in honeybees, see subsequent section.

### POSSIBLE COGNITIVE ASPECTS AND "PLANNING" IN COMB CONSTRUCTION

Comb building capabilities, and the degree of adaptability and individual or collective cognition necessary to achieve the outcome, can be investigated by disrupting or interrupting the normal process. Remarkably visionary experiments on the flexibility of honeybee comb building were described in Huber, (1814/1926) work. Under natural conditions, the comb constructions of cavity-nesting honeybees are attached to the top surface of the cavity and then gradually extended downwards (**Figures 1**, **3**). These bees naturally nest in hollow trees, and therefore typically attach comb to wooden surfaces. To observe a bee colony's inner workings over extended periods, Huber replaced various walls of the hive with glass, and found that when given the choice, bees rejected slippery glass surfaces as starting attachment points for honeycomb construction. When Huber used a glass lid rather than wood for the roof of the hive box, he found that the bees built the honeycomb from bottom to top. The entire building process was thus inverted, with the comb base adhering to the lower horizontal surface, and bees were building cells from the lower side upwards. The upper edge of the comb was curved as it was grown (in the same way as the tip of normal, downward-growing comb is curved). Note that this is far from trivial: the challenge of having a glass ceiling is one that no bee colony would ever have encountered in its evolutionary history. In addition, since the motor routines linked to comb construction are typically aligned with gravity (in the downward direction), bees would have to reverse the contingencies between gravity and the appropriate motor routines in order to build honeycomb of the correct geometry.

Later experiments were designed (Huber, 1814/1926, p. 157) to further coerce the bees into building laterally, achieved by providing a wooden wall but glass roof and floor. Again, the bees were able to adjust their building methods to cope. In that case, they started at one of the side walls and extended the comb laterally across the cavity. It is useful to compare this flexibility with that displayed by other animals whose nest construction has been studied in some detail. Some species of African weaver birds build elaborate all-round enclosed nests that are woven together from grass blades and suspended from tree branches (Walsh et al., 2013). A comparable experiment to that of Huber's would be to prevent weaver birds from access to branches from which to hang their nest; would they be able to build a nest "bottom–up" on a stilt attached to the ground, or one that is at least built directly on the ground? Perhaps they could, but if you further prevent them from using the ground beneath from building their construction, could they suspend a nest between two vertical poles? If they did, you would rightfully conclude that the weaver birds' building behavior is not tightly ruled by hard-wired behavior routines, but that they instead have an awareness of the desirable outcome of their activities, and subjugate their (perhaps partially innate) nest building activities to this outcome. The same interpretation thus should be considered for Huber's findings on bees' building activities.

But Huber's next experiment is perhaps the most remarkable in that it is reminiscent of present day attempts to study animal intelligence by way of their responses to transparent obstacles. In these more recent experiments, a transparent screen is placed between the animal and its target (typically food), and the animal's learning speed in suppressing direct movements to the target (and instead to circumvent the transparent material) is measured (MacLean et al., 2014). This paradigm has been used to compare self-control (as an indicator of cognitive ability) between vertebrate species, though this approach is not without complications (van Horik et al., 2018).

In Huber's experiment, it was not the individual animal's path that had to be adjusted to the appearance of a transparent obstacle, but the trajectory of the growing wax comb. The target in this case was not a food item, but to attach the opposite end of the comb to a suitable vertical surface. After lateral comb construction had begun, Huber placed additional sections of glass to cover the wall toward which the construction was aimed. He anticipated that perhaps once the bees had reached the glass, they would make some sort of special efforts to attach the comb to this suboptimal and slippery surface. But they did something else altogether: apparently noticing that their intended target surface had been rendered suboptimal, the bees took corrective action and turned the construction of their comb by 90◦ — before their construction had reached the target wall (**Figure 4**). Though these experiments are not identical to those designed to test vertebrates' responses, it is noteworthy that no vertebrate displays a spontaneous avoidance of glass obstacles when they are first placed in front of their target; all have to learn from the experience of "bumping into" the obstacles (MacLean et al., 2014).

Huber reported that he repeated this experiment in multiple ways, sometimes moving the glass target into the projected path of their comb building activity several times, and bees would change the direction of their construction again and again. Huber observed that bees had to change the dimensions of the hexagonal

FIGURE 4 | An experiment by Swiss entomologist Huber (1814/1926) to probe the flexibility of the honeybees in comb construction in the face of unusual challenges (computer graphic). Huber had noticed that bees avoid, when possible, to attach the comb construction to glass walls of observation hives. (A) When bees were faced with the hive that had a glass ceiling and floor, they would begin their construction on one of the side walls. (B) When the bees had not yet reached the target wall, a glass screen was placed over that wall. Rather than continuing the construction into the same direction, the bees introduce a curve into the construction by building cells with expanding sizes on the outside of the curve, and cells with reduced orifices on the inside. Continued construction of the comb in the revised direction results in adhesion to a more suitable target area for attachment.

wax cells around the kink; the comb cells on the outside surface were 2–3 times wider than on the inside.

We can dismiss the possibility that bees have an innate response to the glass obstacle between their comb construction site and the intended target wall, since such a challenge has never been encountered by bees in their evolutionary history. Simple learning and memory processes cannot easily explain how an animal copes with wholly novel challenges either, though a non-cognitive explanation of the bees' behavior might begin something like this: since Huber had previously introduced a glass ceiling and roof to the hive box (to force the bees building a laterally expanding comb attached to one of the side walls), the bees had gained experience with glass surfaces and their suboptimal properties in terms of attaching wax. When the new glass screen was inserted on the wall opposite the one where the comb construction had been started, the bees at the front of the construction saw the transparent obstacle whose visual appearance they had previously associated with poor adhesiveness (since Huber's observation hives had a glass lid, it was not dark as in normal beehives, so unlike natural conditions, bees could theoretically have used vision to guide their comb construction). As a result, the bees would have looked around for more suitable target locations to which to attach the comb, and subsequently altered the direction of the expanding comb. In that view, the alteration of the comb construction would be little more than a form of aversive conditioning, where bees simply avoided the glass obstacle that had been placed in their way. Perhaps the construction troupe acted like a swarm of flying birds that is suddenly faced with an obstacle in their path, and took evasive action around the looming stimulus?

There are complications with this "simple" explanation. Even if bees had previously learned to link the visual appearance of glass with poor wax adhesion, it is unclear whether vision would have helped with the solving of the task. Unlike the transparent glass ceiling, the altered target wall is a sheet of glass with a wooden wall behind, and the fact that there was now glass in front of the wood could only have been deduced from subtle mirroring effects (see **Figure 4**). It is uncertain whether bees would be able to see such effects, especially given their poor visuospatial resolution (Spaethe and Chittka, 2003). An alternative that remains to be explored is whether "scouts" assessed the suitability of the target wall by tactile sensing, and then returned from this wall to the construction site, reporting in some way that this wall was no longer suitable. But whether or not the suitability of the target wall is assessed by visual or tactile means, the fact remains that this assessment was done at a distance, before the target wall had been reached — i.e., the bees must have found a way to extrapolate from the current direction of the comb construction to even assess the suitability of the surface to which it might be attached in future, when the comb construction would have advanced further. From Huber's descriptions of the geometry of the experiment, we conclude that the distance at which he introduced the glass screen must have been a minimum of 5 cm (but likely multiple times this) from the tip of the wax construction site. From empirical information about the natural speed of comb construction, it would have taken at least half a day to bridge the remaining distance (Freudenstein, 1961; Hepburn et al., 2014, p. 26). The analogy with the flying bird swarm (in the previous paragraph) thus does not hold in several respects: the wax-constructing bees are not "forward facing" (depending on the current building activity, they might have their heads stuck in partially constructed cells, and of those on the outside of the construction, only a minority will face the direction of the obstacle. Moreover, because of the slow growth of the comb construction, there is no looming stimulus (an obstacle whose apparent size rapidly expands as the subject approaches) – thus there is no simple way to predict the location of contact with the target zone from rapid sensory stimulus change. In addition to predicting the target zone that the construction would have reached many hours or indeed days later, there must have been a process (either visual or tactile) to identify more suitable areas to attach the comb in future, before the direction of building was altered.

At the very least, the following questions must at some level be answered by the comb construction troupe: if we continue building in the current direction, which area of the opposite wall will we reach? Is the surface of this area suitable? If it is not, then what are suitable alternative target areas? After identifying a suitable target area, what is a suitable alteration of current comb building direction to reach that target area in a straight line? A possible cognitive explanation for the bees' collective correction of comb geometry is that there was an appreciation

of the possible (suboptimal) outcome of the construction, were it continued in the initial direction, though this interpretation should be substantiated with further experimentation.

Finally, there remains the question of how the many bees engaged in the construction site agree on changing the direction of the comb. The two basic options are to angle the comb construction to the left or to the right, but more subtle decisions also need to be made: i.e., should the new section of comb be perpendicular to the existing construction, perpendicular to the new target wall, or some oblique angle to either. Whatever the chosen direction, all bees would have to agree; otherwise a lacerated construction would result. That bees (and other social insects) are able to form a consensus among multiple possible options is well-known from the context of searching for, and agreeing on, a new nesting site (Dornhaus and Franks, 2006; Seeley, 2010). However, the heuristics used in this search are related to challenges that have been faced by these insects under natural conditions for millions of years and are therefore shaped by natural selection. Huber's glass wall experiments faced comb-constructing honeybees with a task unprecedented in their history as individuals and as a species. Nonetheless, as a group, they were able to form a consensus for how to best address the challenge. There is, however, a need to replicate these experiments with more detailed recordings of which individuals do what, in the process of assessing suitable target locations for the comb, as well as during the decision making of how to alter the construction.

Natural behavior that appeared to anticipate a need that has yet to arise was also reported by Huber (1814/1926, p. 175). During winter, foraging for flowers, brood rearing, and indeed comb construction is halted, and bees will minimize any activity to ensure that their storage lasts until spring. On one occasion, Huber observed that one of several combs broke off the ceiling of the hive. Not only did bees become active to fortify the dislodged comb with a number of pillars and cross-beams made from wax, but they subsequently also reinforced the attachment zones of all the other combs on the glass ceiling, to ensure that a similar disaster won't happen again. Wrote Huber: "I may restrain myself from reflections and commentaries, but I acknowledge that I could not suppress a sentiment of admiration for an action in which the brightest foresight was displayed." If such anecdotal reports could be verified with multiple replicates under experimental conditions, these results might indeed be examples of prospective cognition or foresight (Clayton et al., 2008; Crystal and Wilson, 2015).

One might counter that the precautionary repairs induced by the mid-winter accidental damage, as well as the responses of bees to Huber's experimental manipulations, might not necessarily be based on foresight – that instead that they might be based on a very large number of hard-wired routines, all triggered by a certain stimulus configuration. This is possible, but one should also consider whether postulating that such a repertoire that includes appropriate responses to every tested experimental manipulation is any more parsimonious than claiming that they do require a form of planning. The challenge would be to explain how such preventive behavioral measures can occur as a result of natural selection. This may be just plausible in the case of preventive midwinter comb fortifications, but it will be very difficult to argue how the anticipatory responses prompted by Huber's experimental manipulations should occur as a result of evolutionary processes — when evolution is very unlikely to have ever presented the kinds of circumstances that Huber faced the bees with. In searching for parsimonious explanations, it is not adequate to use intuitive arguments about which path to the same behavior looks less or more complicated by casual inspection (Chittka et al., 2012). For an evolutionary scenario, one would have to consider which neural circuitry tweaks are necessary for an animal to turn from one that constructs honeycomb by simple robotic principles to one that masters all the unusual challenges above, the mutations that would be required, the environmental conditions that would favor each step. Could it be that a cognitive scenario – where bees have an appreciation of the desired outcome of the comb construction, where behavioral routines are employed relatively flexibly toward reaching the desired goal – could actually be a mechanistically simpler explanation than one that includes a large variety of fixed-action patterns and cognitive tools, including for scenarios that bees won't typically encounter under natural conditions? It is important here to realize that the neuron numbers and circuitries required for agents that can foresee the outcomes of their own actions are certainly not prohibitively large even for insect brains (Shanahan, 2006), and indeed such an ability might have arisen relatively early in evolution as a powerful instrument to solve common but also more unforeseeable challenges in animals' lives (Bronfman et al., 2016).

# CONCLUSION

A traditional idea is that animals have an easily classifiable repertoire of motor routines (in the same way as a Swiss army knife has a limited number of tools with defined functions). For example, perhaps you were taught in school that horses have three gaits (walk, trot, and gallop) and humans two (walk and run). While indeed there may be certain default classes of locomotion in any species, it is clear that humans are capable of an infinity of others — you can crawl, walk on all fours, jump on one leg, walk on crutches, etc. You can easily adapt your locomotion to your current need, your spatial environment, any form of injury, etc. In the same vein, bees by default build hexagonal cells of two dimensions (smaller ones for workers, larger ones for drones), but depending on need, they can also build pentagonal or heptagonal cells, cells that are wider or smaller near the orifice than they are at the base, or use wax for building barriers at the hive entrance to keep out intruders, etc. Huber (1814/1926, p. 178). The historic distinction between behavior being governed by either instinct or learning/cognition is no longer tenable; instead there are interactions at multiple levels and indeed certain instinctual routines that come with an animal's ecological niche will in turn favor certain forms of cognition (Bateson and Mameli, 2007; Audet and Lefebvre, 2017; Robinson and Barron, 2017). For example, all healthy humans have an innate predisposition for language (an 'instinct') (Pinker, 1954/1994), but having the language instinct facilitates almost all cognitive abilities that we

pride ourselves in, including the capacity for cultural evolution, or theory of mind (knowing what others know) (Sacks, 1989). In the same vein, the instinct that determines bees' dietary specialization as consumers of floral nectar and pollen (as opposed to being, e.g., carnivores or parasites) in turn requires them to learn about floral features. We have here dissected a behavior that has been traditionally thought to be wholly governed by instinct. The comb construction abilities demonstrated by honeybees extend beyond a simple algorithm of applying wax to a set pattern; rather, adaptability and error recovery are evident. The insects have a number of perhaps basic, partially hard-wired routines to manufacture the elemental structure of the hexagonal cell (von Oelsen and Rademacher, 1979), but also have the capability to adapt the basic method in order to overcome errors or incompatibilities, to observe and remedy perturbations, to use parts of an elemental cell to correct surface irregularities or to join incompatible sections and, where continued growth would be inadvisable, to take corrective action (Huber, 1814/1926). Huber's classic work suggests that honeybees, rather than building wax comb in the way a robot might, may possess a "master plan" of the desired outcome, and can tailor their efforts to achieve this goal. Such an interpretation is consistent with recent

#### REFERENCES


explorations of intentionality or consciousness-like phenomena in bees (Barron and Klein, 2016; Menzel, 2017; Perry et al., 2017).

#### AUTHOR CONTRIBUTIONS

VG and LC contributed equally to the conception and writing of the content for the review.

#### FUNDING

This work was supported by ERC (Grant No. SpaceRadarPollinator 339347), HFSP Program (Grant No. RGP0022/2014), and EPSRC Program (Grant No. Brains on Board EP/P006094/1).

#### ACKNOWLEDGMENTS

We would like to thank editor Ken Cheng and referees Susan Healy and Zhanna Reznikova for helpful comments on the manuscript.



**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2018 Gallo and Chittka. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Egr-1: A Candidate Transcription Factor Involved in Molecular Processes Underlying Time-Memory

Aridni Shah<sup>1</sup> \*, Rikesh Jain<sup>2</sup> and Axel Brockmann<sup>1</sup> \*

<sup>1</sup> Tata Institute of Fundamental Research, National Centre for Biological Sciences, Bengaluru, India, <sup>2</sup> School of Chemical and Biotechnology, SASTRA University, Thanjavur, India

In honey bees, continuous foraging is accompanied by a sustained up-regulation of the immediate early gene Egr-1 (early growth response protein-1) and candidate downstream genes involved in learning and memory. Here, we present a series of feeder training experiments indicating that Egr-1 expression is highly correlated with the time and duration of training even in the absence of the food reward. Foragers that were trained to visit a feeder over the whole day and then collected on a day without food presentation showed Egr-1 up-regulation over the whole day with a peak expression around 14:00. When exposed to a time-restricted feeder presentation, either 2 h in the morning or 2 h in the evening, Egr-1 expression in the brain was up-regulated only during the hours of training. Foragers that visited a feeder in the morning as well as in the evening showed two peaks of Egr-1 expression. Finally, when we prevented time-trained foragers from leaving the colony using artificial rain, Egr-1 expression in the brains was still slightly but significantly up-regulated around the time of feeder training. In situ hybridization studies showed that active foraging and time-training induced Egr-1 up-regulation occurred in the same brain areas, preferentially the small Kenyon cells of the mushroom bodies and the antennal and optic lobes. Based on these findings we propose that foraging induced Egr-1 expression can get regulated by the circadian clock after time-training over several days and Egr-1 is a candidate transcription factor involved in molecular processes underlying time-memory.

Keywords: Egr-1, honey bee foraging, time-memory, anticipation, small Kenyon cells

# INTRODUCTION

Honey bee foraging has been one of the most fruitful behavioral paradigms in the study of sensory and cognitive capabilities of insects and animals in general (von Frisch, 1967; Giurfa, 2007; Chittka, 2017). Foragers continue to visit a highly rewarding food source for days and weeks till it gets exhausted. This persistent behavior enables researchers to train honey bee foragers to an artificial sugar-water feeder which then can be used as a tool for psychological experiments (Chittka et al., 1999; Wagner et al., 2013). For example, presenting the feeder at a specific time during the day showed that honey bees learnt the time of food presentation and demonstrated for the first time that animals have a sense of time (Beling, 1929; Wahl, 1932, 1933).

Since then, many behavioral studies followed, investigating foraging entrainment (= time of food presentation shifts behavioral/physiological rhythms which may/may not reflect a true timeplace association) and time memory (= ability of individual foragers to associate the presence of

#### Edited by:

Martin Giurfa, UMR5169 Centre de Recherches sur la Cognition Animale (CRCA), France

#### Reviewed by:

Charlotte Helfrich-Förster, Universität Würzburg, Germany Darrell Moore, East Tennessee State University, United States

#### \*Correspondence:

Aridni Shah aridnis@ncbs.res.in Axel Brockmann axel@ncbs.res.in

#### Specialty section:

This article was submitted to Comparative Psychology, a section of the journal Frontiers in Psychology

Received: 21 March 2018 Accepted: 14 May 2018 Published: 05 June 2018

#### Citation:

Shah A, Jain R and Brockmann A (2018) Egr-1: A Candidate Transcription Factor Involved in Molecular Processes Underlying Time-Memory. Front. Psychol. 9:865. doi: 10.3389/fpsyg.2018.00865

**72**

food with both location and time of day) (Wahl, 1932; Koltermann, 1971; Moore et al., 1989; Naeger et al., 2011). Time-memory experiments showed that honey bee foragers are capable of associating food related cues like odor, color or spatial location with time (Gould, 1987; Zhang et al., 2006; Pahl et al., 2007; Prabhu and Cheng, 2008) and can memorize up to nine different feeder times per day (Koltermann, 1971). There is convincing evidence that daily foraging entrainment of bees and time-memory are regulated by the circadian clock (Renner, 1955, 1957, 1959; Beier, 1968; Frisch and Aschoff, 1987; Bloch, 2010; Fuchikawa et al., 2017).

Recently, we showed that continuous foraging is accompanied by a sustained up-regulation of the immediate early gene Egr-1 (early growth response protein-1; see Chen et al., 2016; Duclot and Kabbaj, 2017) and candidate downstream genes involved in learning and memory (Singh et al., 2017). Our results indicated that up-regulation of Egr-1 is dependent on the food reward. Now, we were interested in the question whether timetraining over several days might affect the expression of Egr-1. Behaviorally, time-training of honey bee foragers leads to anticipatory activity (Moore et al., 1989; Moore and Doherty, 2009), thus it could be possible that time-training might also lead to an anticipatory molecular response. We performed a series of different time-training experiments similar to those that have been done before (Wahl, 1932; Moore et al., 1989; Naeger et al., 2011) but instead of testing the behavioral responses, we measured Egr-1 expression on a test day at which the food reward was not presented.

Our experiments showed that Egr-1 expression is highly correlated with the time and duration (hours) of feeder training even when the food reward is not presented. Foragers visiting a feeder over the whole day showed up-regulated Egr-1 expression throughout the day, whereas foragers trained to visit a feeder for only a few hours in the morning or in the evening showed higher expression only during the respective training time. Foragers trained to visit two feeders at different times of the day showed two peaks of Egr-1 up-regulation. Most importantly, foragers that were prevented from leaving the colony still showed a slight but significant up-regulation of Egr-1 around the time of feeder training. These results suggest that Egr-1 expression might get regulated by the circadian clock after time-training over several days. Up-regulation of Egr-1 in the artificial rain experiments could be interpreted as an anticipatory molecular response. This conclusion is supported by the fact that the spatial expression pattern of Egr-1 in the brain induced during foraging or activated in the artificial rain experiments, were very similar. We propose that Egr-1 represents a candidate molecular link between the output of the circadian clock and the learning and memory systems involved in foraging.

#### MATERIALS AND METHODS

#### Animal

Apis mellifera colonies were purchased from a local beekeeper and kept in an outdoor flight cage (12 m × 4 m × 4 m) on the campus of the National Centre for Biological Sciences, Bangalore, India. The colonies consisted of 4-frames within a standard commercial wooden hive box, each frame containing approximately 2500–3000 bees. The day-night length as well as the temperature conditions inside the flight cage were similar to the natural conditions in Bangalore with an approximate 12-h light-dark cycle all throughout the year. During the experimental period from November to February the time of sunrise changed from 06:15 to 06:45 and that of sunset changed from 17:50 to 18:30. The flight cage was devoid of any flora and the only food source available was the feeder provided by the experimenter.

#### Training Regime

The training regime consisted of presentation of a colored plate with sugar-water solution (1 M) without odor, unless mentioned otherwise, for 10 days. We trained the bees for 10 days (a) since we need enough bees for the experiment (practical restriction) which can be achieved by more days of training since more bees are recruited (b) to increase temporal accuracy for the trained time, since Moore and Doherty (2009) show that with increase in the number of days, temporal accuracy increases. The duration and time of presentation differed according to the experiment. After the training time, the sugar-water was washed off and the plate kept back at the feeder location, to avoid association of the feeder plate with the food reward. Foragers trained to the feeder were marked at the feeder on the 7th day and the collections, as documented for each experiment below, were done on the 11th day.

## Collection Without Food Reward

Honey bee foragers were allowed to forage ad libitum (all day) or trained to forage in the morning (08:00 to 10:00) or in the evening (16:00 to 18:00). On the 11th day, the food reward was not added to the feeder plate. Marked foragers were collected at six different time-points at 4 h intervals: 06:00, 10:00, 14:00, 18:00, 22:00, and 02:00. Foragers were collected from the hive, which involved opening the hive and temporarily removing the comb frames from the hive. Each experiment was performed on a separate colony.

The bees were immediately flash frozen in liquid nitrogen and stored at −80◦C until further processing for RNA isolation.

# Collection of Bees Trained to 2 Feeders

As a pilot experiment, we trained foragers to 2 different feeders within a day that were separated in space and time in December 2017. The colonies were placed in a flight cage that was longer, but had the same height and width (24 m × 4 m × 4 m). The food sources were placed at the opposite ends, almost 24 m apart from each other. One feeder was blue colored with 1 M sucrose scented with 20 ul Phenylacetaldehyde per liter of sucrose and the other feeder was green colored with 1 M sucrose scented with 20 ul Linalool per liter of sucrose. The blue feeder was opened in the morning, from 08:00 to 10:00 whereas the green feeder was opened in the evening from 16:00 to 18:00. The feeder plates were left in their position after the training time. On the 5th day of training, foragers coming to each feeder were individually marked with numbered tags. From the 7th day onward, observations were made, and the bees were classified into

groups that continuously visited either the morning or evening or both the feeders.

On the 11th day, bees that visited both the feeders were collected at 06:00, 09:00, 13:00, 17:00, 22:00 and 02:00. The timepoints were chosen such that the bees were caught at 60 min after the start of the feeder training for both the feeders (09:00 and 17:00) and at an intermediate time-point (13:00). The other timepoints were chosen according to the single feeder experiment, "collection without food reward" mentioned above.

Bees that visited either the morning or the evening feeder, were collected only at the 2 trained time-points (09:00 and 17:00). Since many of the marked bees were lost by the 11th day, only three bees per time-point were successfully analyzed.

This experiment was repeated in February 2018 with the training duration reduced to 1 h, i.e., 08:00 to 09:00 in the morning and 17:00 to 18:00 in the evening, in order to increase the separation between the 2 feeder times. From the 8th day onward, foragers coming to each feeder were marked with paint marks on the thorax in the morning and the abdomen in the evening. Each day was marked with a different color. On the 11th day, collections were done as per the above-mentioned time-points. Marked foragers were collected from the hive, which involved opening the hive and temporarily removing the comb frames from the hive. Feeder visits of the bees shown in Supplementary Figure S2.

#### Collection With "Artificial Rain" Setup

Honey bee foragers were trained to forage either in the morning (08:00 to 10:00), afternoon (12:00 to 14:00), or evening (16:00 to 18:00) during the months of December 2016, February 2017 and November 2017, respectively. On the 11th day, the "artificial rain" setup was started at 06:00. The "artificial rain" setup consisted of a box made of plexiglass which would hold water, called the water basin. The water flowed continuously through pores on the underside of the water basin. The entire setup was positioned such that the hive entrance was completely blocked by the falling water and hence prevented the bees from flying out. Any marked bee that crawled out of the hive and escaped, was caught and chilled on ice until the completion of the collections. Collections were made from 5 equidistant holes on the inner lid of the hive that was covered with a black chart paper. The holes had a flap cover which was opened at the time of collection and a 50 ml tube was placed over the hole. Bees that crawled up the tubes, being attracted to light, were chilled on ice and then the marked foragers were separated out and flash frozen. This collection method was adopted to prevent bees from flying out during collections. Since we were interested in the expression pattern in and around the trained time, we collected bees at half hour intervals starting from an hour before the trained time up until an hour after the trained time. The rest of the time-points corresponded to previous collection time-points.

Collection time-points for morning trained bees: 06:00, 07:00, 07:30, 08:00, 08:30, 09:00, 10:00, 14:00, 18:00, 22:00. Collection time-points for noon trained bees: 06:00, 10:00, 11:00, 11:30, 12:00, 12:30, 13:00, 14:00, 18:00, 22:00. Collection time-points for evening trained bees: 06:00, 10:00, 14:00, 15:00, 15:30, 16:00, 16:30, 17:00, 18:00, 22:00.

# Brain Dissection, RNA Isolation, cDNA Preparation and Quantitative PCR

Frozen brains were dissected on a dry ice platform in a glass cavity block in 100% ethanol. Brains were homogenized in TRIzol (Invitrogen, Life Technologies, Rockville, MD, United States) using a motorized homogenizer and RNA was extracted using the standard Trizol-chloroform method. Glycogen (20 mg/ml, Thermo Scientific, Life Technologies, Rockville, MD, United States) was added for increased recovery of RNA. cDNA was prepared using the SuperScriptTM III First-Strand Synthesis System (Invitrogen, Life Technologies, Rockville, MD, United States).

Primers for Egr-1 and RP49 qPCR were the same that were used in Singh et al. (2017). Egr-1 primers recognise a region in exon 3, hence amplify all 3 isoforms of Egr reported by Sommerlandt et al. (2016). Primers for Cry-2 were Forward: 5<sup>0</sup> -AGGTCTCACATACTCTTTACA-3<sup>0</sup> ; Reverse: 5 0 -ACTGTTGGTACTGGTGGT-3<sup>0</sup> . The qPCR was performed following the same protocol as in Singh et al. using Kapa SybrGreen (KapaBiosystems, Wilmington, Massachusetts, United States). The standard curve method was followed and RP49 was used as the internal control.

## cDNA Cloning

To generate riboprobes for Egr-1, primers (Forward: 5<sup>0</sup> - AAAGGGAGAGAGAGGATGAAG-3<sup>0</sup> ; Reverse: 5<sup>0</sup> -TAATGC GGTGGTGTGAGTTC-3<sup>0</sup> ) were generated to amplify a 1096 bp fragment of the gene in exon 3. RNA was isolated and converted to cDNA following the procedure as described above and the cDNA was used as a template to amplify the fragment. The fragment was then purified using PCR Purification kit (Qiagen, Hilden, Germany) followed by cloning of the fragment into the pCRTMII-TOPO <sup>R</sup> vector using the TOPO TA Cloning Kit (Invitrogen, Life Technologies, Rockville, MD, United States) following the manufacturer's protocol. The cloning mix was transformed into E. coli (DH5-alpha) and screened using the blue-white screening regime. The plasmids were then isolated and sequenced for confirming the presence and orientation of insert.

#### RNA in Situ Hybridization

Time-trained "active foragers" were collected from the feeder at 60 min after the onset of foraging. Time-trained "non-active foragers" were caught at 60 min after onset of the training time from the hive using the "artificial rain" setup. Time-trained control bees were caught at 6 h before the trained time from the hive. The bee brains were freshly dissected on DEPC water and immediately embedded into the Jung Tissue Freezing Medium (Leica Microsystems, Nussloch, Germany). The embedded brains were then sectioned using a HYRAX C-25 cryostat into 12 µm thin sections and collected on Superfrost Plus Microscope slides (Fisherbrand, Hampton, NH, United States). The slides were allowed to dry at room temperature for about 10 min and kept on dry ice until further processing.

RNA probes were synthesized using SP6 Polymerase or T7 Polymerase using DIG RNA labeling mix (Roche, Indianapolis, IN, United States) incubated for 2 h at 37◦C. The probes were then purified using the Qiagen Micro kit (Qiagen, Hilden, Germany) and stored at −80◦C.

fpsyg-09-00865 June 1, 2018 Time: 13:27 # 4

The slides were fixed in 4% PFA overnight at 4◦C. The slides were washed for 20 min in 0.1 M phosphate buffer (PB), followed by treatment with 10 mg/ml Proteinase K solution for 15 min at room temperature (RT), re-fixation in 4% PFA for 15 min at 4◦C, followed by treatment with 0.2 M HCl for 10 min and 0.25% acetic anhydride in TEA for 10 min. Each step was followed by a 5 min wash with 0.1 M PB. The slides were dehydrated through an ethanol gradient of 70% → 95% → 100% and airdried for 1 h. The slides were pre-hybridized in the hybridization buffer (50% formamide, 10 mM Tris–HCl pH 7.6, 200 ug/ml tRNA, 1X Denhardt solution, 10% Dextran sulfate, 600 mM NaCl, 0.25% SDS, 1 mM EDTA) without riboprobes for 1 h at 60◦C. The riboprobes were added to the hybridization buffer followed by denaturation at 85◦C for 5 min. The denatured probes were added to the slides and allowed to hybridize overnight at 60◦C in a mineral oil bath.

The slides were then washed through a series of SSC buffer, starting with 5X SSC (rinse), 1:1 solution of formamide and 2X SSC for 30 min at 60◦C, followed by 2X SSC and 0.2X SSC for 20 min each at 60◦C and finally, 3 washes with TNT (0.1 M Tris, 0.15 M NaCl, 0.05% Tween) at RT.

For detection, the slides were first blocked with 5% BSA for 30 min at RT, followed by incubation in Anti-DIG POD (Roche, Indianapolis, IN, United States) overnight at 4◦C. After incubation, slides were washed in TNT and incubated in Tyramide-Cy5 (Perkin Elmer, MA, United States) for 15 min followed by washing and mounting with Vectashield with DAPI (Vector Laboratories, CA, United States). The fluorescent images were captured using Olympus FV1000 at a magnification of 10 × with 1 um thick optical sections. Post hoc adjustments of brightness and intensity were made using ImageJ analysis software (NIH, United States).

#### Statistics

All statistical analyses were performed using R [R 3.4.1 GUI 1.70 El Capitan build (7375)] (R Core Team, 2017). Since the data-points were not normally distributed, Kruskal Wallis (KW) tests were done. When the KW-test was significant, post hoc analyses for comparison amongst the groups was done using the dunn.test package in R (Dinno, 2017) with p-values adjusted for multiple comparisons using the Benjamini-Hochberg ("bh") method (Benjamini and Hochberg, 1995). The alpha was set at 0.05. All data are represented as box-plots with individual data-points indicated.

#### RESULTS

## Restricted Time-Training Leads to Time-Restricted Egr-1 Up-Regulation

Honey bee foragers were allowed to forage ad libitum or were trained either to a morning feeder (08:00 to 10:00) or to an evening feeder (16:00 to 18:00) for 10 days. On the 11th day, the marked foragers were collected from the hive in the absence of food reward. Honey bees that foraged at the ad libitum feeder showed elevated expression of Egr-1 throughout the day with highest expression at 14:00 (**Figure 1A**, p-values in Supplementary Table S1). Foragers that were trained to a morning or an evening feeder showed significant up-regulation in the mRNA levels of Egr-1 at the time of feeder training, i.e., 10:00 in the morning trained and 18:00 in the evening

trained foragers compared to most of the other time-points (**Figures 1B,C**, p-values in Supplementary Tables S2, S3). The time point directly following the training time in the morning experiment and the one preceding the training time in the evening experiment showed p-values that were slightly above the cut off (morning 14:00: p = 0.06; evening 14:00 p = 0.07). Restricted foraging for a short time of the day led to a restricted Egr-1 expression occurring only around the time of training.

# Individual Foragers Trained to 2 Feeders at Different Times of the Day Showed 2 Peaks of Egr-1 Expression

Next, we tested the dynamics of Egr-1 expression in individual bees trained to 2 different feeders separated in space and time. Both feeders differed in color and odor and one was opened in the morning while the other was opened in the evening. Presenting a colony with 2 feeders at different times of the day resulted in 3 foraging groups: (a) bees that visited only the morning feeder ("only morning"), (b) bees that visited only the evening feeder ("only evening"), and (c) bees that visited both the feeders ("both feeder").

Egr-1 brain expression levels of the bees that visited only one feeder showed a peak at the time they had been trained to visit the feeder similar to our previous experiments (**Figure 2A**, KW test: ns; **Figure 2C**, p-values in Supplementary Table S4). In contrast, Egr-1 expression levels of the bees that visited both feeders showed two peaks, one at each training time (**Figure 2B**, KW test: ns; **Figure 2D**, p-values in Supplementary Table S5). When the bees were trained to visit the feeders for only 1 h starting at 08:00 and 17:00, expression of Egr-1 was significantly downregulated at the intermediary time-point of 13:00 (**Figure 2D**). In the 2 h training experiment, the Egr-1 expression was not down

FIGURE 2 | Egr-1 expression in individuals exposed to two feeders. (A,B) Bees were trained for 2 h each in the morning and evening. (A) Those bees that visited only the morning feeder (blue) or the evening feeder (red) showed comparatively higher expression in the morning or evening, respectively, however, not significant. (B) Bees that visited both the feeders (green) showed comparatively higher expression at both the time-points. The time-point in between the 2 training times (13:00) showed a down-regulation trend, however, none of the time-points were significantly different. n = 3 per time-point since enough bees could not be caught. (C,D) Experiment was repeated with 1-h training period each to increase separation between the training times. (C) Bees that visited the morning (blue) feeder showed significantly higher expression at 09:00 compared to the "morning only" bees at 18:00 as well as "evening only" bees at 09:00. Similarly, "evening only" (red) bees showed significantly higher expression at 18:00 compared to "evening only" bees at 09:00 as well as "morning only" bees at 18:00. (D) Bees that visited both feeders (green) showed significantly higher expression at both the trained time-points compared to all other time-points. 13:00 showed significantly lower levels of Egr-1; n = 5 per time-point. Data shown as relative expression changes compared to the lowest value per group in the form of box-plots with individual data-points delineated. KW-test with Dunn's ("bh" method) multiple comparison was done for single feeder visiting bees ("only morning" + "only evening") and "both feeder" visiting bees, p-values are shown in Supplementary Tables S4, S5, respectively.

regulated at 13:00 (**Figure 2B**). A greater temporal separation of the training period led to a more distinct regulation of Egr-1 expression.

# Foragers Visiting "Both Feeders" Showed Cry-2 Expression Similar to "Evening Only" Bees

Naeger et al. (2011) showed that morning and evening trained foragers differ in the expression pattern of Cry2 and Per indicating that the foragers likely developed different circadian rhythms according to their foraging activity. Therefore, we got interested in the question how foragers visiting one or two feeders over the day differ in Cry2 expression. In the "morning only" bees, Cry-2 expression levels at 09:00 and 18:00 were very similar, whereas in the "evening only" bees, Cry-2 expression levels were significantly higher at 09:00 compared to 18:00 (**Figure 3**, p-values in Supplementary Table S6). These results are consistent with those of Naeger et al. (2011) who showed a similar expression pattern for Cry-2, when they trained bees from 09:00 to 10:15 or from 17:00 to 18:15 and looked at transcripts levels at the 2 trained time-points for both the groups. The bees that visited "both feeders" in our experiments showed a Cry2 expression pattern similar to the "evening only" foragers, with significantly higher expression at 09:00 compared to 18:00.

Different to the earlier experiments, in which the training time was forced onto the bees, bees in our experiments could choose when to forage and this decision then influenced their circadian clock.

#### Egr-1 Got Up-Regulated in Time-Trained Foragers Prevented From Flying Out

To test whether Egr-1 expression in time-trained foragers is regulated by the circadian clock, we tested Egr-1 expression in time-trained foragers that were prevented from flying out. If Egr-1 is under the influence of the circadian clock, it would be upregulated at the trained time even in the absence of flight activity and other environmental cues. To reduce any stress responses that might occur in bees mechanically restricted from leaving the colony, we used an "artificial rain" setup (Supplementary Figure S1A) (Riessberger and Crailsheim, 1997). As in the case of natural rain, the foragers would not fly out. The feeder-trained foragers were collected from the hive.

Morning trained foragers prevented from flying out showed a slight but significant up-regulation of Egr-1 about an hour before the trained time, i.e., at 07:00 and the up-regulation was maintained till the end of training time with a peak at 08:30. The expression levels dropped after the trained time, and at 18:00 the expression was significantly lower compared to the highest level of Egr-1 expression i.e., 08:30 and hence had dropped to levels equivalent to 06:00 (**Figure 4A**, p-values in **Table 1**).

In the afternoon trained foragers, Egr-1 showed an expression pattern similar to morning trained foragers with significant elevation at 11:00 compared to the 10:00 and 06:00. The elevated expression was maintained till the end of training time with a peak at 12:30 and then dropped significantly by 18:00 (**Figure 4B**, p-values in **Table 2**).

In the evening trained foragers, the Egr-1 expression was very low in the morning, with no difference in levels at 06:00, 10:00 and 22:00. An up-regulation trend was observed at 14:00, however, it was not statistically significant. A statistically significant up-regulation was observed at 15:00, and the up-regulation was maintained till the end of training time with a peak at 15:30. The expression levels dropped down to minimum values at 22:00 (**Figure 4C**, p-values in **Table 3**; Supplementary Figure S1B, p-values in Supplementary Table S7).

Together, our artificial rain experiments clearly showed that Egr-1 expression is up-regulated in time-trained foragers without any foraging or flight activity. This molecular response resembles anticipatory behavior of time-trained honey bee foragers (Moore et al., 1989).

FIGURE 4 | Egr-1 expression when the bees were prevented from flying out using "artificial rain" setup. (A) Bees that were trained from 08:00 to 10:00, already showed significant up-regulation of Egr-1 by 07:00 and remained up-regulated till the end of training time with a peak at 08:30. The mRNA levels started to decline at 09:00 and was reduced significantly by 14:00 and remained low for the rest of the day. (B) Bees that were trained from 12:00 to 14:00, showed significant up-regulation by 11:00 with a peak at 12:30. The expression declined thereafter and was significantly low by 18:00. (C) Bees that were trained from 16:00 to 18:00 showed an up-regulation trend already by 14:00, although not significant. mRNA levels were significantly increased by 15:00 with a peak at 15:30 which started to decline thereafter, differing from the trends seen in morning and afternoon trained bees. Data shown as relative expression changes compared to 06:00 in the form of box-plots with individual data-points delineated, n = 5. KW-test with Dunn's ("bh" method) multiple comparison was done on each experiment, p-values are shown in Tables 1–3, respectively.

### "Active Foragers" and Time-Trained "Non-active Foragers" Show Egr-1 Expression in the Same Population of Mushroom Body Cells (Small Kenyon Cells)

To identify the brain regions that could be involved in foraging and time-training related Egr-1 up-regulation, we performed brain in situ hybridization for Egr-1. Specifically, we compared the Egr-1 expression pattern of brains from actively foraging honey bees caught 60 min after the onset of foraging ("active foragers") (see Singh et al., 2017) and time-trained but not flying foragers caught 60 min after onset of the training time ("nonactive foragers"). As a control we used foragers caught from the hive, 6 h before the training time.

The control bees showed very low staining with only a few cells in the antennal lobes stained (**Figures 5A–C**). In "active foragers", predominant expression of Egr-1 was seen in the small Kenyon cells (sKCs) compared to large Kenyon cells (lKCs), where only few cells showed staining (**Figures 5D–F**, **6A**). "Nonactive foragers" also showed Egr-1 expression in the sKCs. The expression was lower compared to "active foragers" and more specifically expressed in the sKCs. Very few lKCs were stained in the "non-active foragers" suggesting anticipatory up-regulation of Egr-1 specifically in the sKCs (**Figures 5G–I**, **6B**). We limited our analysis to the mushroom bodies, because they allow a clear identification and comparison of neuron populations between different individuals. Our stainings suggest that there might be additional neuron populations in other brain areas involved in these processes.

TABLE 1 | Adjusted p-values for Artificial Rain Experiment (08:00-10:00 trained).


p-values less than 0.05 shown in bold.

fpsyg-09-00865 June 1, 2018 Time: 13:27 # 8

TABLE 2 | Adjusted p-values for Artificial Rain Experiment (12:00-14:00 trained).


p-values less than 0.05 shown in bold.

TABLE 3 | Adjusted p-values for Artificial Rain Experiment (16:00-18:00 trained).


p-values less than 0.05 shown in bold.

# DISCUSSION

The major finding of our study is that time-restricted foraging and feeder time-training over several days led to time-restricted Egr-1 daily expression pattern. Foragers that visited one feeder for a restricted time period showed one peak of Egr-1 expression, whereas those that visited two different feeders at two separate times of the day showed highest expression at the 2 trained time-points. Even more importantly, time-trained foragers that were prevented from flying out showed significant Egr-1 expression around the time of training indicating that training time is sufficient to induce Egr-1 up-regulation. These experiments suggest that bees respond to time-training not only with anticipatory behavior but also an anticipatory molecular response. Egr-1 is already slightly up-regulated in expectation of a food reward.

Based on these and earlier results, we propose that Egr-1 expression is regulated by foraging associated food reward as well as the circadian clock after several days of time-training. We cannot comment upon acquisition of memory or the expression profile of Egr-1 in the initial days of training since we have not tested it. It is possible that Egr-1 is up-regulated after a single day of training but the temporal accuracy of expression might be affected similar to the anticipatory behavior (Moore and Doherty, 2009).

In "active foragers," Egr-1 is expressed in the cells of the mushroom bodies (MB), optic lobes (OL), and antennal lobes (AL). Since MBs are thought to be involved in learning and

FIGURE 5 | In situ hybridization of Egr-1 on brains of foragers. (A–C) Trained bees that were collected 6 h before the trained time from the hive showed very low expression of Egr-1, with only a few cells in the antennal lobes stained. (D–F) "Active foragers", collected from the feeder at 60 mins past the start of foraging time, showed strong Egr-1 expression in the mushroom bodies as well as other brains parts like antennal lobes and optic lobes. (G–I) Time-trained "non-active foragers," collected from the hive with the "artificial rain"setup at 60 min past the trained time, showed specific expression only in the small Kenyon cells. MB, mushroom bodies; OL, optic lobes; AL, antennal lobes.

FIGURE 6 | Focus on the mushroom bodies of the "active foragers" and the "non-active foragers". (A) Almost all the small Kenyon cells (white stars) are stained for Egr-1 whereas only some of the large Kenyon cells (yellow stars) that are closer to the calyces show staining in the "active foragers". (B) "Non-active foragers" showed very specific staining of the small Kenyon cells (white stars) and a few cells close to the "lip" region of the calyces only. Li, Lip; Co, Collar; BR, Basal Ring.

memory processes (Mizunami et al., 1998; Hourcade et al., 2010; Lefer et al., 2012), we focused for now, on the expression in the MBs. Within the MBs, the small Kenyon cells (sKCs) showed predominant staining whereas only some of the lKCs closer to the calyces showed Egr-1 expression (**Figure 6A**). In "non-active foragers," Egr-1 was expressed in the sKCs only (**Figure 6B**).

Therefore, the sKCs may play a crucial role in foraging related time-memory. Interestingly, some of the candidate downstream targets of Egr-1 (Khamis et al., 2015) that showed significant expression during foraging (Singh et al., 2017), namely, Hr38 (Yamazaki et al., 2006), EcR (Takeuchi et al., 2007), and DopR2 (McQuillan et al., 2012) have been shown to be specifically expressed in the sKCs.

Although there is some information about differences in developmental origin, sensory inputs as well as the expression of particular genes between the different Kenyon cell types, we do not know anything about their functions and a functional separation among them.

The sKCs form a central cluster directly located above the basal ring, which their dendrites innervate. The basal ring receives multiple sensory inputs from optic lobes, in particular the lobula, (Ehmer and Gronenberg, 2002), antennal lobes (Gronenberg, 2001), and the suboesophageal ganglia (Schröter and Menzel, 2003). Farris et al. (1999) showed that the sKCs are the last Kenyon cells to be generated during development and proposed that they might be involved in the MB growths at the nurseforager transition.

The lKCs are separated in a central cluster that innervates the collar and an outer cluster that innervates the lip region of the calyces. The collar receives inputs only from the visual system and the lip only from the olfactory system (Strausfeld, 2002; Farris, 2013). Given the differences in the sensory inputs, it is tempting to speculate that the sKCs might have a unique function in foraging related, i.e., food reward induced, learning processes and time-memory. In contrast, Lutz and Robinson (2013) reported a pronounced Egr-1 expression in the lKCs during orientation flights, which precede foraging and are independent of food reward.

The expression pattern of Cry2 showed that "morning only" and "evening only" foragers have different Cry2 expression patterns, suggesting that they are on different circadian time schedules. Bees foraging at "both feeders" in the morning and the evening showed an expression pattern similar to "evening only" bees although they were foraging in the morning and the afternoon. So far, bee chronobiologists have made a distinction between entrainment and non-entrainment time memory models (Moore, 2001). The entrainment model proposes that the clock oscillator will get entrained to the time of the food presentation which then shifts behavioral or physiological rhythms similar to changes in the light/dark cycle. The non-entrainment model hypothesizes that a representation of the circadian phase at which a foraging experience occurred is stored together with features of the food source in a separate memory system (Moore and Doherty, 2009). Both mechanisms might not necessarily exclude each other and could act in parallel (Mistlberger, 1994). The results of our 2-feeder experiment actually support the idea, that both processes might be intertwined. Foraging entrainment affects the cycling of clock genes (master oscillator), and timememory could be based on an association of Egr-1 expression and a specific phase of the clock cycling (memory of oscillator phase).

Frisch and Aschoff (1987) clearly demonstrated that timerestricted feeder presentation under constant light/dark cycle leads to an entrainment of a colonies' foraging activity. So far nothing is known about the sensory channel and respective clock neurons in the brain involved in this foraging entrainment. There are two plausible mechanisms, either foraging entrainment is based on an independent food entrainable oscillator (FEO) or foraging entrainment modulates some part of the canonical light entrainable oscillator (LEO) master clock. As honey bees are dependent on a time-compensated sun compass for navigation, information of the light/dark cycle is highly likely present in foraging entrained foragers.

The artificial rain experiments demonstrated that Egr-1 expression is initiated in time-trained foragers at least an hour before the training time. In the evening trained foragers, this up-regulation trend appears to start 2 h before the trained time, although not significant. These expression patterns fit with previous work on anticipatory flight behavior that demonstrated that bees trained in the morning and afternoon show shorter anticipatory flight activity whereas those trained in the evening show longer anticipatory flight activity (Moore et al., 1989). Therefore, Egr-1 could be a molecular equivalent of anticipatory behavior.

Based on our studies, we propose a model for Egr-1 function in honey bee foraging (**Figure 7**):

turn regulates the expression of downstream targets that are involved in learning and memory. (b) Time-Restricted foraging at one food source leads to entrainment of the molecular clock. This effect might be restricted to a specific population of clock cells. For example, different populations of clock cells might be involved in foraging entrainment and time-compensated sun compass navigation. (c) Time-training over several days leads to an anticipatory up-regulation of Egr-1 that is controlled by the circadian clock. Thus, Egr-1 expression in the Kenyon cells of the mushroom bodies might be regulated via two signaling mechanisms, one from food reward related pathways and one from the circadian clock.


#### REFERENCES


# AUTHOR CONTRIBUTIONS

AS and AB designed the experiments of the study and wrote the manuscript. AS and RJ performed the experiments and analyzed the data.

# FUNDING

This study was supported by NCBS-TIFR institutional funds to AS and AB (12P4167). RJ was supported by an ICMR fellowship.

### ACKNOWLEDGMENTS

We thank Adrianna Schatton for helping to establish in situ hybridization protocol in the lab, and Manal Shakeel for help with fluorescent imaging and Central Imaging & Flow Cytometry Facility (CIFF), NCBS for confocal imaging.

#### SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpsyg. 2018.00865/full#supplementary-material



**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2018 Shah, Jain and Brockmann. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Omega-6:3 Ratio More Than Absolute Lipid Level in Diet Affects Associative Learning in Honey Bees

Yael Arien<sup>1</sup> , Arnon Dag<sup>2</sup> and Sharoni Shafir<sup>1</sup> \*

<sup>1</sup> B. Triwaks Bee Research Center, Department of Entomology, The Robert H. Smith Faculty of Agriculture, Food and Environment, The Hebrew University of Jerusalem, Rehovot, Israel, <sup>2</sup> Gilat Research Center, Institute of Plant Sciences, Agricultural Research Organization, Negev, Israel

Floral pollen is a major source of honey bee nutrition that provides them with micro- and macro-nutrients, including proteins, fatty acids, vitamins, and minerals. Different pollens vary in composition, including in the essential fatty acids, alpha-linolenic acid (omega-3) and linoleic acid (omega-6). Monocultures, prevalent in modern agriculture, may expose honey bee colonies to unbalanced omega-6:3 diets. The importance of omega-3 in the diet for adequate learning and cognitive function, with a focus on suitable omega-6:3 ratio, is well documented in mammals. We have recently shown, for the first time in invertebrates, the importance of omega-3 in diets for associative learning ability in honey bees. In the current work, we examine the effect of the absolute amount of omega-3 in diet compared to the omega-6:3 ratio on honey bee associative learning. We fed newly emerged bees for 1 week on different artificial diets, which had lipid concentration of 1, 2, 4, or 8%, with omega-6:3 ratios of 0.3, 1, or 5, respectively. We then tested the bees in a proboscis-extension response olfactory conditioning assay. We found that both omega-6:3 ratio and total lipid concentration affected learning. The most detrimental diet for learning was that with a high omega-6:3 ratio of 5, regardless of the absolute amount of omega-3 in the diet. Bees fed an omega-6:3 ratio of 1, with 4% total lipid concentration achieved the best performance. Our results with honey bees are consistent with those found in mammals. Best cognitive performance is achieved by a diet that is sufficiently rich in essential fatty acids, but as long as the omega-6:3 ratio is not high.

Keywords: Apis mellifera, omega-6, omega-3, cognition, conditioning, nutrition

# INTRODUCTION

Honey bees (Apis mellifera) are social insects that live in highly organized colonies, consisting of a queen, many workers, and some drones. Division of labor among the workers is age-dependent (Winston, 1987). Young bees mostly work inside the colony, whereas older bees engage in foraging. Honey bee foraging behavior, as well as other characteristics of the honey bee, makes them the most important pollinator in commercial crops (Klein et al., 2007) providing important contributions to human nutrition (Chaplin-Kramer et al., 2014). Bees require floral nectar and pollen for their nutrition. Nectar is the main source of carbohydrates and pollen provides micro- and macro-nutrients, including proteins, fatty acids (FA), vitamins, and minerals. Bees prefer to collect

#### Edited by:

Martin Giurfa, UMR5169 Centre de Recherches sur la Cognition Animale (CRCA), Université Toulouse III Paul Sabatier, France

#### Reviewed by:

Fabien Pifferi, UMR CNRS/MNHN 7179, Muséum National d'Histoire Naturelle, France Mathieu Lihoreau, Centre National de la Recherche Scientifique (CNRS), Université Toulouse III Paul Sabatier, France

> \*Correspondence: Sharoni Shafir Sharoni.Shafir@mail.huji.ac.il

#### Specialty section:

This article was submitted to Comparative Psychology, a section of the journal Frontiers in Psychology

Received: 01 March 2018 Accepted: 30 May 2018 Published: 19 June 2018

#### Citation:

Arien Y, Dag A and Shafir S (2018) Omega-6:3 Ratio More Than Absolute Lipid Level in Diet Affects Associative Learning in Honey Bees. Front. Psychol. 9:1001. doi: 10.3389/fpsyg.2018.01001

**84**

pollen from a variety of plants (Avni et al., 2009). Moreover, colony performance is affected by the quality and quantity of pollen that the colony consumes and high lipid levels in pollen was found to promote honey bee health (Di Pasquale et al., 2013). Starvation and malnutrition were rated as the second main reason, after poor quality queens, for colony loss in the United States (Hayes et al., 2008). There is therefore growing interest in research of honey bee nutrition (Manning, 2016; Démares et al., 2017; Corby-Harris et al., 2018).

Fatty acid contents and composition in pollen varies between different types of plants (Roulston and Cane, 2000). In modern agriculture, beehives are frequently placed in large monoculture areas, where bees forage on single pollen. This may lead to a diet that is unbalanced in its essential components, such as amino and fatty acids, which could lead to malnutrition (Naug, 2009). Fatty acids are the main component in cell membranes and are important for their function. They are necessary for reproduction and development, serve as a source for energy and for the development of fat bodies in bees during winter (Kunert and Crailsheim, 1988; Manning, 2001).

Most fatty acids can be synthesized endogenously according to the body's needs. Fatty acids that the body cannot produce must be provided through nutrition, accordingly they are called essential fatty acids (EFAs). Two groups of EFAs are omega-3 and omega-6, which are polyunsaturated fatty acids (PUFAs) (Simopoulos, 1991). Alpha-linolenic acid (ALA) and linoleic acid (LA) are the major omega-3 and omega-6 fatty acids, respectively, found in pollen, though their abundance differs between different pollen species (Manning, 2001). In mammals, both EFAs can be elongated to long chain PUFAs, LA to arachidonic acid (AA) and ALA to eicosapentaenoic acid (EPA) and docosahexaenoic acid (DHA). Those are the dominant EFAs in mammals and can be obtained in the diet mainly through fish oil and marine algae (Simopoulos, 2009).

Because of the low prevalence of omega-3 in the modern western diet, most researches have focused on the impact of deficiency of this FA. In mammals, the importance of omega-3 fatty acids is well known. These fatty acids constitute a major proportion of total FAs of brain, retina, and sperm in humans and other mammals (O'Brien et al., 1964). Deficiency of omega-3 fatty acids, mainly long-chain PUFAs, is associated with increase in frequency of chronic diseases, poor health and especially with several mental and cognitive disorders (Yashodhara et al., 2009; Gow and Hibbeln, 2014). The nutritional effects of deficiency in omega-3 in insects were investigated for the first time in honey bees (Arien et al., 2015). Bees that were fed the low omega-3 diets had great decrease in olfactory and tactile associative learning. These findings showed, that similar to mammals, omega-3 fatty acids have a crucial role for cognitive function of honey bees.

However, it is debated in the mammalian (including human) literature as to the relative detrimental effect of omega-3 deficiency as opposed to a high omega-6:3 ratio. The modern Western diet, for example, is biased toward omega-6, with omega-6:3 ratio of about 15:1, whereas this ratio in traditional human diets was about 1:1 (Simopoulos, 2009). In mammals, LA and ALA can be desaturated through enzymes to long chain PUFAs. Not only that this conversion process is very slow (Chow, 2000), but also there is competition between omega-6 and omega-3 fatty acids on the affinity to the desaturation enzymes. There are two enzymes (delta-4 and delta-6 desaturases) with greater affinity to omega-3 over omega-6 (Insua et al., 2003; Bazan, 2006). However, a high intake of LA interferes with the desaturation and elongation of ALA (Patterson et al., 2012).

A similar question arises in honey bee nutrition: whether bees require a particular absolute amount of omega-3 or to maintain a particular omega-6:3 ratio. Insects have only trace amounts of long-chain PUFAs (Shen et al., 2010). The dynamics between LA and ALA may be different than those in mammals. However, the findings of a very strong effect on bee cognition of ALA deficiency (unlike mammals in which cognitive impairment results from EPA and DHA deficiency), raises the hypothesis that there may be important LA:ALA dynamics in bees that affect bee cognition and health.

The primary aim of this research was to test whether the cognitive impairment in honey bees is due to low absolute amounts of omega-3 in the diet or to a high omega-6:3 ratio. In making diets that differed in these two factors, also the total lipid levels varied. A second aim, therefore, was to test the effect of total lipid levels on cognitive performance. Newly emerged bees were fed for a week diets that differed in omega 6:3 ratio and total lipids levels and were then tested in an olfactory conditioning test.

#### MATERIALS AND METHODS

#### Diets

In order to control the fatty acids composition in the bees nutrition, we fed them artificial diets. As in previous experiments, we used soy flour as the source for protein (Arien et al., 2015). However, this flour also contains fatty acids, with the dominant one being omega-6 (LA), making it difficult to control omega-6:3 ratios. Therefore, we created diets based on flour after an oil extraction process, using a soxhlet system. Hexane at 70◦C was used to extract residues of oil from the flour for 6 h, and was then evaporated to obtain fat-free flour. The protein contents of the soy flour was 47% protein (analyzed by the Kjeldahl method; see Arien et al., 2015), and was added as to achieve a 20% protein diet. The composition of the diets was: 42% fat-free soy flour, between 49.5 and 56.5% honey, which contains negligible amount of lipids (Machado De-Melo et al., 2018), and 1–8% mixture of two vegetable oils: flax and corn. Flax oil is 97% fatty acids, and is rich in omega-3, whereas corn oil is 90% fatty acids, and is rich in omega-6. The relative amount of each oil varied between treatments to achieve the desired FA composition (see Arien et al., 2015 for FA analyses of these oils). **Table 1** shows the EFA composition of the diet treatments. There were four groups of treatments with different levels of percentage of lipids in the diet: 1, 2, 4, and 8%, and within each group the ratio of omega-6 to omega-3 oils was 5, 1, or 0.3. The diets were designed so that we could compare the same three levels of omega-6:3 ratio in four levels of lipid concentration and with different absolute omega-3 amounts. We could thus compare the learning ability of bees fed diets that varied in omega-6:3 ratio but had similar absolute omega-3 amounts and we could compare bees fed diets


TABLE 1 | The experimental diets by their lipid percentage and omega-6:3 ratio, the essential fatty acid composition of the total fatty acids (TFA) and absolute amounts.

that varied in absolute omega-3 amounts while maintaining the same omega-6:3 ratio.

The essential FAs comprised between 54 and 65% of the TFA. The relative composition of the two essential FAs varied most, in comparison to the common non-essential FAs, between the different omega-6:3 ratio diets (Supplementary Table S1).

#### Bees

Bees were of the local strain of honey bees, which is based mostly on the Italian bee, Apis mellifera ligustica. We placed sealed brood combs from ordinary hives in an incubator overnight. The following day we randomly collected up to 24 h-old bees that emerged in the incubator and placed them inside 9-cm petri-dishes with filter paper at the bottom, in groups of five bees. To each petri-dish we added two 1-ml Eppendorf feeders, one with diet and one with water. The bees were fed one of the diet treatments for 1 week, as in Arien et al. (2015); pollen consumption is mostly by young bees during the first days after emergence (Crailsheim et al., 1992). Diet consumption per dish was calculated by weighing the feeders at the beginning and after the 1 week in the incubator, taking into account weight loss due to evaporation by having for each diet a control dish with no bees. The diets contained honey so there was no need for supplemental carbohydrates. Then the bees were taken for olfactory conditioning of the proboscis-extension response (PER) experiments. There were between 31 and 34 bees in each treatment in PER experiments.

# Olfactory Proboscis-Extension Response (PER) Conditioning

Proboscis-extension response experiments were preformed according to established methods (Bitterman et al., 1983; Drezner-Levy et al., 2009). The experiment was conducted in a temperature-controlled laboratory with AC set at 26◦C (range was 24–29◦C). The petri-dish with the bees was placed in a freezer for 3–5 min, and then the immobilized bees were restrained into 5-cm long pieces of drinking straws by attaching duct tape around the sectioned top part of the straw and the thorax of the bee. Forty-five minutes later all bees were fed 1 µl of 50% (w/w) sucrose. One hour after the feeding we tested the bees for their appetitive motivation. We touched the antennae of each bee with a cotton stick soaked in 50% (w/w) sucrose solution; the bees were not fed during this test. Bees that did not extend their proboscis were removed from the experiment. Twenty motivated bees, which did extend their proboscis, were taken for conditioning and mounted along rulers in haphazard order. The experiment started immediately after the motivation test. The odors used in this experiment were Benzyl acetate and Geranyl acetate. To provide the odors we placed a strip of filter paper inside a glass syringe tube and dripped on it 3.5 µl of pure odor. The syringe was connected to an air pump controlled by computer. The odor was delivered for 4 s followed by provisioning of a reward for 3 s. The bees were exposed to the two odors in 12 conditioning trials, 6 to each odor, with an inter-trial interval of 8 min. One odor was associated with a positive reward (odor A) and the other odor with a negative reward (odor B). Odors were presented in a pseudorandom sequence ABBABAABABBA. Following presentation of odor A, the bees were fed by a Gilmont micro syringe 0.4 µl 50% sucrose solution as a positive reward (CS+). Following presentation of odor B, the negative reward (CS−) consisted of touching the antennae with a cotton-stick dipped in a 2 M NaCl solution (the bee was not fed the salt solution).

#### Statistical Analyses

To test the effect of diets on learning performance we calculated a learning index, which was the difference between the sum of responses in the three last trials to the CS+ and CS− (Shafir and Yehonatan, 2014). We used a two-way ANOVA to test the effect of the omega 6:3 ratio and percentage of lipids in diets as main factors, their interaction and hive number as random variable, on the learning index. All statistics were done using the JMP v.13 software (SAS Institute).

#### RESULTS

Mean diet consumption per dish was not affected by the omega 6:3 ratio (F2,<sup>300</sup> = 0.37, P = 0.69), nor by the total lipid concentration (F3,<sup>300</sup> = 1.603, P = 0.19), and the interaction

between these two factors was not statistically significant (F6,<sup>300</sup> = 0.6, P = 0.73). Comparison of the weekly diet consumption between all 12 treatments is shown in **Figure 1**.

Learning performance was significantly affected by diet omega-6:3 ratio (F2,<sup>386</sup> = 17.9, P < 0.0001) and total lipid concentration (F3,377.<sup>5</sup> = 2.96, P = 0.03), with the interaction between the two factors not being significant (F6,<sup>385</sup> = 0.79, P = 0.58). **Figure 2A** shows the learning curves of bees fed diets that differed in omega-6:3 ratio, pooling together all total lipid concentrations. Such comparison shows that learning performance of bees fed diets with omega-6:3 ratio of 5 was significantly lower than of those fed diets with lower omega-6:3 ratios of 1 and 0.3. When presented with sucrose solution, almost all bees imbibed it in almost all the trials, and there was no difference in the US response between groups (F2,<sup>386</sup> = 0.04, P = 0.96) (**Figure 2B**). **Figure 3A** shows the learning curves of bees fed diets that differed in total lipid concentration, pooling together all omega-6:3 ratios. Lowest performance was of bees fed diets with 1% lipids, and best performance was of bees fed diets with 4% lipids. The percentage of lipids in diets also did not affect the US response (F3,377.<sup>5</sup> = 0.36, P = 0.78) (**Figure 3B**). Comparison of learning indexes between all 12 treatments is shown in **Figure 4**. The learning index of bees fed omega-6:3 ratio of 5 was consistently the lowest within all lipids groups. Bees which had omega-6:3 ratio of 1 in their diets with 4 and 2% lipids, achieved the highest learning scores (**Figure 4**).

#### DISCUSSION

In the present paper, we study the effect of omega-6:3 ratio and lipids content in honey bee nutrition on learning ability. Our results are consistent with our previous findings (Arien et al., 2015), that learning performance in bees is greatly impaired by a diet deficient in omega-3 and high in omega-6:3 ratio. Here, however, we experimentally separated the effect of absolute levels of omega-3 from that of the omega-6:3 ratio. We found that a minimal total absolute amount of EFAs is required, but thereafter the main effect on learning performance is of the omega-6:3 ratio. Specifically, high omega-6:3 ratio impairs learning, even if the absolute amount of omega-3 in the diet is relatively high. Bees fed diets that had lower omega-6:3 ratio (1 or 0.3) learned better than those fed diets with high omega-6:3 ratio (of 5), even when the absolute amount of omega-3 was similar.

In human nutrition, a high omega-6:3 ratio has been associated with cognitive decline in adults (Loef and Walach, 2013). The amount of omega-6 consumed can modulate the amount of omega-3 FAs and thereby reduce the amount of omega-3 available in the body (Taha et al., 2014). Adults that performed poorly in cognitive tests had higher ratio of omega-6:3 FAs in their blood plasma compared to those that performed better (Cherubini et al., 2007).

Andruchow et al. (2017) found a correlation in healthy older adults between dietary ratio of omega-6:3 and spatial cognition; those whose diet contained a lower omega-6:3 ratio had better spatial memory and performed better in navigation

FIGURE 3 | Average of performance in olfactory conditioning of bees according to percentage of lipids in the diet. (A) Learning curves show the proportion of bees that extended their proboscis to the conditioned odor in each of six trials with each odor. The full lines show learning curves to a positively rewarded conditioned stimulus (CS+). The dashed lines represent the negative rewarded conditioned stimulus (CS−). (B) Shows the response to the sucrose reward, unconditioned stimulus (US+). Different letters represent statistically significant difference between treatments (Tukey HSD test, P < 0.05).

tests. Similarly, rats fed lower omega-6:3 ratio diets performed better in a task requiring navigating a maze (Hajjar et al., 2012). Similarities between bees and mammals in the detrimental effects of high dietary omega-6:3 ratio on learning performance suggests that similar to mammals, also bee spatial cognition might be impaired by high omega-6:3 ratio. Honey bees have sophisticated navigation and orientation abilities, which are crucial for colony survival (Collett et al., 2013). Colony collapse disorder (CCD), for example, involves bees departing the colony and failing to return to if for yet unknown reasons (Oldroyd, 2007; Traynor et al., 2017). The effect of high omega-6:3 ratio diet on honey bee navigation and spatial learning deserves further study.

In a study in which pollen was collected by hand from 28 different plant species, the range of omega-6:3 ratio was between 0.09 and 5.34 (Arien et al., 2015). The highest values were of Eucalyptus trees. Our results suggest that a colony situated in a Eucalyptus monoculture forest would suffer from this high omega-6:3 ratio. There are other crops that are grown in monocultures, and which have relatively high omega-6:3 ratio pollen, which are probably not ideal for a colony. However, when a colony is situated in a habitat with diverse vegetation, it tends to collect pollen from several plants at the same time (Avni et al., 2009). The omega-6:3 ratio of pollen mixtures collected by bees in several places around the world ranged between 0.3 and 0.9 (Arien et al., 2015), which is in the optimal range for cognitive functions according to our results.

It is debated whether honey bee foragers can assess the nutritional value of pollen, especially with regards to its protein contents (reviewed by Zarchin et al., 2017). However, in choice experiments, Hendriksma and Shafir (2016) recently showed that honey bee foragers preferred to collect diets that balanced their nutritional deficiencies, including in essential amino acids and EFAs. Furthermore, when a colony was fed pollen lacking a specific EFA, foragers attempted to compensate for this deficiency at the colony level by evaluating complementary pollen as more attractive in their recruitment dances (Zarchin et al., 2017). Thus, it appears that a honey bee colony needs a balanced omega-6:3 diet, and that it attempts to selectively forage so as to achieve it. The geometric framework approach to nutrition (Simpson and Raubenheimer, 2012) has been applied lately to assess the macronutrient requirements of honey bees, for example the balance between proteins and carbohydrates (Paoli et al., 2014; Helm et al., 2017). We are presently using this approach to further evaluate the omega-6:3 requirements of honey bees.

Bumblebee foragers prefer a protein to lipids (P:L) ratio of between 5:1 and 10:1 (Vaudo et al., 2016a,b). Interestingly, callow honey bees in our study consumed equal amounts of all diets (**Figure 1**), though the P:L ratio of our diets ranged between 20:1

and 2.5:1, for the 1 and 8% lipid diets, respectively. These young bees, during the first week of their life, are the main consumers of pollen in the colony (Crailsheim et al., 1992). It appears that young honey bees, until the age of 1 week, may be focused on protein and not be regulating lipid consumption.

Learning performance was affected also by the diet lipids content; bees fed a low-fat diet of 1% lipids had the lowest learning curve, regardless of the omega-6:3 ratio. Thus, even a diet whose EFA contents was strongly biased toward omega-3 (with omega-6:3 ratio of 0.3), could not support good learning when the total lipid content (and therefore the absolute amount of omega-3) was too low. The reported range of pollen lipids content is between 2 and 20%; the range is reduced to between 3 and 8% for bee bread, which consists of a mixture of several pollens stored in cells within the hive (Wright et al., 2017). In this study, we show that for good learning ability lipid levels should be between 2 and 8% with peak performance at 4% total lipids. Relatively high pollen lipid concentration is also important for proper brood development. Di Pasquale et al. (2013) found that young nurse bees developed well when fed several pollens with total lipid contents of between 6.4 and 7.4%.

Impairment of olfactory associative learning may have direct adverse consequences to the functioning of a honey bee colony (Klein et al., 2017). In the present study, we tested the effect of a pollen-substitute diet on the performance of 8-day-old bees. The typical task of young bees at this age is to be nurses, which attend to and feed the larvae (Page and Robinson, 1991). Many of the social interactions between adult bees and between nurse bees and larvae depend on chemical signaling (Amdam, 2011). We have previously shown that the impaired learning performance due to omega-3 deficiency could not be attributed solely to impairment of olfactory perception, as tactile associative learning was equally affected (Arien et al., 2015). We are currently testing specifically whether olfactory perceptual abilities are also affected by omega-6:3 imbalances. It remains to be determined how impairment of olfactory perception and/or of olfactory associative learning would impact the ability of nurse bees to raise larvae.

Since older bees hardly consume pollen any more, we assume that the detrimental cognitive effects accumulated over the first week of life would persist into older age. We in fact found severe learning deficits in older bees from a colony fed an omega-3 deficient diet (Arien et al., 2015). The typical task of older

#### REFERENCES


bees is foraging, a task that requires sophisticated cognitive abilities. Foragers need to quickly learn to associate between floral attributes and nectar and/or pollen rewards (Menzel, 1999), and between floral attributes and predation risk (Abbott and Dukas, 2009). Thus, the learning impairments conferred by nutritional deficits of callow bees is likely to adversely affect the foraging behavior and survival of older bees.

Honey bees have provided an exceptionally rich model for comparative cognition (Menzel, 2012; Giurfa, 2015; Perry et al., 2017). We have previously shown that as in mammals, omega-3 deficiency severely impaired honey bee associative learning (Arien et al., 2015). Here, we strengthen this finding and furthermore show that, as hypothesized for mammals, learning performance is mostly affected by dietary omega-6:3 ratio. The honey bee may prove a useful model for comparative studies of the nutritional basis of cognitive performance.

#### AUTHOR CONTRIBUTIONS

YA, AD, and SS designed the experiments. YA and SS wrote the paper with help from AD and analyzed the data. YA performed the experiments.

# FUNDING

Financial support was provided by a BBSRC grant (BB/P007449/1), Vatat Nehemia Levtzion fellowship to YA, and internal funds of the B. Triwaks Bee Research Center.

#### ACKNOWLEDGMENTS

We would like to thank Haim Kalev for beekeeping and field assistance, Maor Zavitan and Shiran Yona for laboratory assistance.

#### SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpsyg. 2018.01001/full#supplementary-material



honey: a review. J. Apic. Res. 57, 5–37. doi: 10.1080/00218839.2017.133 8444



**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

The reviewer ML and handling Editor declared their shared affiliation.

Copyright © 2018 Arien, Dag and Shafir. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Maggot Instructor: Semi-Automated Analysis of Learning and Memory in Drosophila Larvae

#### Urte Tomasiunaite<sup>1</sup> , Annekathrin Widmann1,2 \* † and Andreas S. Thum1,3 \* †

<sup>1</sup> Department of Biology, University of Konstanz, Konstanz, Germany, <sup>2</sup> Department of Molecular Neurobiology of Behavior, Georg-August-University Göttingen, Göttingen, Germany, <sup>3</sup> Department of Genetics, University of Leipzig, Leipzig, Germany

For several decades, Drosophila has been widely used as a suitable model organism to study the fundamental processes of associative olfactory learning and memory. More recently, this condition also became true for the Drosophila larva, which has become a focus for learning and memory studies based on a number of technical advances in the field of anatomical, molecular, and neuronal analyses. The ongoing efforts should be mentioned to reconstruct the complete connectome of the larval brain featuring a total of about 10,000 neurons and the development of neurogenic tools that allow individual manipulation of each neuron. By contrast, standardized behavioral assays that are commonly used to analyze learning and memory in Drosophila larvae exhibit no such technical development. Most commonly, a simple assay with Petri dishes and odor containers is used; in this method, the animals must be manually transferred in several steps. The behavioral approach is therefore labor-intensive and limits the capacity to conduct large-scale genetic screenings in small laboratories. To circumvent these limitations, we introduce a training device called the Maggot Instructor. This device allows automatic training up to 10 groups of larvae in parallel. To achieve such goal, we used fully automated, computer-controlled optogenetic activation of single olfactory neurons in combination with the application of electric shocks. We showed that Drosophila larvae trained with the Maggot Instructor establish an odor-specific memory, which is independent of handling and non-associative effects. The Maggot Instructor will allow to investigate the large collections of genetically modified larvae in a short period and with minimal human resources. Therefore, the Maggot Instructor should be able to help extensive behavioral experiments in Drosophila larvae to keep up with the current technical advancements. In the longer term, this condition will lead to a better understanding of how learning and memory are organized at the cellular, synaptic, and molecular levels in Drosophila larvae.

Keywords: Drosophila larvae, aversive olfactory conditioning, optogenetics, olfactory receptor neurons, electric shock, mushroom body

# INTRODUCTION

Various technical and conceptual successes have helped recent research to gradually understand how a brain organizes learning and memory. Although, we still cannot understand and address a number of basic mechanisms, recent achievements are fascinating. Part of this development is due to the work on less complex insect brains, such as that of the fruit fly Drosophila and its larva

#### Edited by:

Martin Giurfa, UMR5169 Centre de Recherches sur la Cognition Animale (CRCA), France

#### Reviewed by:

Teiichi Tanimura, Kyushu University, Japan Dennis Mathew, University of Nevada, Reno, United States

#### \*Correspondence:

Annekathrin Widmann annekathrin.widmann@unigoettingen.de Andreas S. Thum andreas.thum@uni-leipzig.de †These authors have contributed equally to this work.

#### Specialty section:

This article was submitted to Comparative Psychology, a section of the journal Frontiers in Psychology

Received: 21 March 2018 Accepted: 31 May 2018 Published: 20 June 2018

#### Citation:

Tomasiunaite U, Widmann A and Thum AS (2018) Maggot Instructor: Semi-Automated Analysis of Learning and Memory in Drosophila Larvae. Front. Psychol. 9:1010. doi: 10.3389/fpsyg.2018.01010

**92**

(Heisenberg, 2003; Gerber and Stocker, 2007; Gerber et al., 2009; Busto et al., 2010; Diegelmann et al., 2013; Waddell, 2013, 2016; Cognigni et al., 2017; Widmann et al., 2017).

The benefits that the Drosophila larva offers for the analysis of learning and memory are based on several factors. First, the elementary organization of the larval central nervous system consists of only about 10,000 neurons (Dumstrei et al., 2003; Nassif et al., 2003). Second, the availability and robustness of behavioral assays that also allow to specifically address distinct memory phases (Aceves-Pina and Quinn, 1979; Scherer et al., 2003; Widmann et al., 2016). Third, the existence of transgenic techniques, which allow manipulation of neuronal networks, small sets of neurons, or even individually identified neurons (Luan et al., 2006; Pfeiffer et al., 2010; Li et al., 2014). Especially the establishment of a large set of single-cell split-Gal4 lines specific for the larval memory center – the mushroom body (MB) – has to be highlighted (Saumweber et al., 2018). Finally, the establishment of the larval connectome that includes the reconstruction of every individual neuron with all its synapses and synaptic partners (Ohyama et al., 2015; Berck et al., 2016; Jovanic et al., 2016; Schlegel et al., 2016; Eichler et al., 2017). These advantages now allow, for the first time, projects that can purposefully investigate – by using thousands of newly established genetic tools – how learning and memory are organized at the level of the brain, the nerve cell and the synapse.

The study of large amounts of different transgenic animals is simplified by the use of automated methods for behavioral research. However, in contrast to the adult Drosophila, these techniques are unavailable for the analysis of learning and memory in larvae (Colomb et al., 2009; Schnaitmann et al., 2010; Aso and Rubin, 2016; Ichinose and Tanimoto, 2016). The majority of behavioral learning assays in use are based on the principle of classical conditioning (aka Pavlovian conditioning) (Pavlov, 1927). In such studies, a biologically active stimulus (e.g., appetitive stimulus: food; aversive stimulus: electric shock), the unconditioned stimulus (US), is paired with a previously neutral stimulus (e.g., an odor), the conditioned stimulus (CS).

For almost 40 years (Aceves-Pina and Quinn, 1979), standard assays have been used on agar or agarose-filled Petri dishes and are very robust, easy to learn, inexpensive and require no complex technology (Gerber and Stocker, 2007). At the same time, however, such assays are time-consuming and laborintensive, as the larvae have to be manually transferred to different Petri dishes during the entire experiment. In total, depending on the applied training regime, the conditioning of one group of animals using standard assays requires an average of 45–60 min. Consequently, this condition makes standard assays suitable to a limited extent for use in large behavioral screens. However, given the establishment of thousands of different genetic tools manipulating precisely the larval brain at the cellular and molecular level, such screens are becoming more important (Li et al., 2014; Saumweber et al., 2018). To use these resources extensively for larval learning and memory research, behavioral experiments or at least parts of them should be automated.

Thus, we designed the Maggot Instructor, a device to train Drosophila larvae in an automated fashion. The applied behavioral protocol uses electric shock as US paired with the artificial activation of a single olfactory receptor neuron (ORN) as CS (instead of a real odor). Drosophila larvae receive olfactory stimuli via the dorsal organ, a single sensillum located on the right and left sides of the head, with each housing 21 ORNs (Singh and Singh, 1984; Oppliger et al., 2000; Fishilevich et al., 2005; Kreher et al., 2005). For a specific odor, the dedicated ORNs or combinations of ORNs perceive the respective sensory information and signal it further to the larval main olfactory center – the antennal lobe (AL) (Fishilevich et al., 2005; Kreher et al., 2005; Ramaekers et al., 2005). All ORNs connect directly in a one-to-one fashion to 21 uniglomerular projection neurons (PNs). Most of the uniglomerular PNs in turn are directly connected to single-claw Kenyon cells (KC) in the MB calyx region (Eichler et al., 2017). Therefore, for almost every input channel, a direct connection from an ORN (first order) to a PN (second order) to a KC (third order) exists. As a consequence, optogenetically, individual ORN input channels can be activated to generate odor-specific learning and memory in the MB via simultaneous application of a US (Honda et al., 2014). However, in addition to this labeled line pathway, 14 additional multiglomerular PNs exists and initially about 100 KCs (in young L1 larvae) are randomly associated to two or more PNs (Berck et al., 2016; Eichler et al., 2017). These neurons can process odor information at different levels in a more integrative fashion.

To artificially activate the defined neurons, sophisticated optogenetic methods, which benefit from the semitransparent cuticle of the larvae, have been introduced (Schroll et al., 2006; Dawydow et al., 2014; Rohwedder et al., 2015). By using a twopart expression system, such as the Gal4/UAS system (Brand and Perrimon, 1993), proteins like channelrhodopsin2 (ChR2) or its improved variant ChR2-XXL (Schroll et al., 2006; Dawydow et al., 2014), a light-activated cation channel, can be possibly expressed to depolarize neurons by blue light in a time-wise precisely controlled manner. Single-cell specificity for ORNs can be achieved by using an established set of Or-Gal4 lines that use different Or promoter gene fragments to direct Gal4 expression to individual neurons (Fishilevich et al., 2005). Double-activation learning and memory experiments also become possible by replacing sugar reward (the US) by thermogenetic activation of octopaminergic (OA) neurons with the dTrpA1 channel and odor stimuli (the CS) by optically activating an ORN with ChR2 (Honda et al., 2014). This experiment is feasible as OA and dopamine (DA) neurons mediate sugar reward information in the larval brain (Selcho et al., 2014; Rohwedder et al., 2016; Saumweber et al., 2018). By contrast, the perception of electric shock by the Drosophila larva remains unelucidated. However, the DA system is also sufficient and necessary for aversive olfactory learning and memory in the larvae (Selcho et al., 2009). Four DA neurons innervating the vertical lobe, the lateral appendix, and the lower peduncle of the MB are possibly crucial for signaling aversive stimuli (Eichler et al., 2017).

The current model suggests that during training, a certain pattern of KCs activated by an odor (or in our case by artificial activation by light) occurs simultaneously with a modulatory

signal about the aversive or appetitive US mediated by different sets of DA neurons (Heisenberg, 2003; Waddell, 2013, 2016). Coincident activation of KCs will in turn change the synaptic connectivity of KCs onto extrinsic MB output neurons (MBONs). Thus, during learning, MBONs change their response properties and act as odor-specific neurons that report the presence of a particular odor as an alerting signal for the conditioned behavior. The Maggot Instructor automates this step by executing the behavioral training protocol independently in a high-throughput manner.

# MATERIALS AND METHODS

#### Fly Stocks (Keeping and Crossing)

Fly strains were reared on standard Drosophila medium at 25◦C in complete darkness. Or42b-Gal4 (Bloomington Stock No: 9972), Or47a-Gal4 (Bloomington Stock No: 9982), UAS-ChR2- XXL (Bloomington Stock No: 58374) and w<sup>1118</sup> (obtained from R. Stocker) were used. Strains crossed with w<sup>1118</sup> served as controls. For all the behavioral experiments, the flies were transferred to new vials and allowed to lay eggs for 2 days. Third instar feeding-stage larvae aged 96–144 h were used for behavioral experiments

#### Assay Plates and Odors

Petri dishes (85 mm diameter; Cat. No. 82.1472, Sarstedt, Nümbrecht) were used as the test plates, as described previously (Pauls et al., 2010b; Huser et al., 2012, 2017; Gerber et al., 2013). The test plates and training chambers were filled with 2.5% agarose (Sigma-Aldrich, Cat. No. A9539, CAS No. 9012-36-6). In several behavioral experiments 0.01 M lithium chloride (Sigma-Aldrich, Cat. No. 298328, CAS No. 85144-11-2) was mixed with 2.5% agarose. Throughout the test, the Petri dishes were covered with perforated lids for an equal distribution of odors. All the experiments were performed at about 21◦C. As olfactory stimuli in the test we used 10 µl amyl acetate (AM, Sigma-Aldrich, Cat. No. 46022; CAS No. 628-63-7; diluted at 1:100, 1:250, 1:500, 1:600 and 1:750 in paraffin oil, Sigma-Aldrich, Cat. No. 76235, CAS No. 8012-95-1), benzaldehyde (BA, undiluted; Sigma-Aldrich, Cat. No. 12010, CAS No. 100-52-7) and ethyl acetate (EA, Sigma-Aldrich, Cat. No. 270989; CAS No. 141-78-6; diluted 1:1000 in paraffin oil, Sigma-Aldrich, Cat. No. 76235, CAS No. 8012-95-1). Odorants were loaded into custom-made Teflon containers (4.5-mm diameter) with perforated lids (Scherer et al., 2003) and were used for no longer than 5 h after preparation.

### Experimental Setup/Compact Real-Time Input Output (cRIO)

The Maggot Instructor consists of a training box wired with a computer that controls the type and timing of the applied stimuli via a cRIO system and an automated training device (ATD) (Graetzel et al., 2010; Kain et al., 2012; Dylla et al., 2017). cRIO (NI 9074) from National Instruments was used as a controlling device for the automated training protocol. cRIO was also used to regulate and monitor the technical aspects, such as the fine adjustment of parameters (e.g., light intensity, voltage, or temperature). The software Build Digital Output Sequence with Frequency Output (BDOS) was used for programming cRIO (Dylla et al., 2017). All settings in cRIO were transmitted to the training box (see below), where the parameters, including electric shock or light intensity were adjusted appropriately. Larval training was carried out in an elongated metal box (the training box), which was separated into 10 training chambers with the same size and can be regulated in parallel or individually. Each chamber consists of a case with an electrode at the front and rear end, a Peltier element underneath the chamber and odor inlets and outlets on all four sides. The training chamber is closed by a lid, which contains a white and a blue LED.

### Training Protocol

Only L3 larvae that are in the feeding stage were used. This requirement was achieved by collecting the larvae from the top layer only of the food substrate. Ten groups with 30 larvae each were collected, washed with tap water, and stored in a water drop for up to 30 min before the experiment. To avoid artificial activation of ORNs in the experimental animals, these steps were performed under red light. Before the experiment, the training chambers were filled with 2.5% agarose to cover the entire bottom with a substrate layer of about 1 cm thickness. After the preparation, the larvae were transferred to the training chambers. The larvae from every genotype were used in each run. For several runs, the training chambers were consistently varied for each genotype. Several runs were possible per training chamber with the same agarose substrate. To prevent the larvae from escaping the training chambers, a custom-made plastic frame covered with a plastic net was inserted into each training chamber. This technique was established by Khurana et al. (2009). This method also prevented the larvae from climbing the training chamber and thus avoiding electric shock. The training chambers were also moistened with about 1 ml of tap water to ensure the proper hydration of the larvae. Afterward, the lids of each training chamber and the cover of the Maggot Instructor were closed. The device was switched on, and the previously defined training protocol was started. All the training steps including CS (if not otherwise mentioned at a light intensity of about 86,000 lux) and US (if not otherwise mentioned electric shock of 120 V) application, then ran automatically. The training lasted for 60 min.

After training, the cover of the Maggot Instructor and the lids of each training chamber were removed. For the test, the larvae from each training chamber were placed on a fresh, pure agarose assay plate with an odor container on the one side and a second container without olfactory cue on the other side. The sides were randomly changed for every training chamber. All the larvae from one training chamber located on the plastic frames and the agarose cover bottom were collected and transferred. The larvae were placed in the center of the Petri dish, the lid was closed, and the larvae were given 5 min to freely move on the test plate. Ten test plates were analyzed in parallel (one for each training chamber). A Preference Index was calculated by subtracting the number of larvae on the control container side (CC) from the number of larvae on the odor side (ODOR) and dividing the result by the total number of larvae on both sides and in the middle zone (TOTAL):

Preference Index = (#ODOR – # CC)/#TOTAL (1)

The positive values indicate attraction to the odor, whereas the negative values represent aversion.

#### Statistical Analysis

fpsyg-09-01010 June 18, 2018 Time: 16:9 # 4

All data processing, statistical analyses, and visualizations were conducted with GraphPad Prism 7.0a. Figure alignments were performed with Adobe Photoshop CC. The groups that showed no violation of the assumption of normal distribution (Shapiro–Wilk test) and homogeneity of variance (Bartlett's test) were analyzed with parametric statistics. One-way ANOVA was applied followed by planned pairwise comparisons between the relevant groups with a Tukey's honestly significant difference post hoc test (comparisons between groups larger than two). Experiments with data that significantly differed from the assumptions above were analyzed with the non-parametric Kruskal–Wallis test followed by Dunn's multiple pairwise comparison. To compare single genotypes against chance level, we used one sample t-test or Wilcoxon signed-rank test. The significance level of statistical tests was set to 0.05. Data were presented as box plots, with 50% of the values of a given genotype being located within the boxes and the whiskers representing the entire set of data. Outsiders are indicated as dots. The median performance index was indicated as a bold line and the mean as a cross within the box plot.

#### RESULTS

#### Maggot Instructor: A Custom-Made, Automated Approach to Train Larvae

A comprehensive set of standardized behavioral assays is available to analyze learning and memory in Drosophila larvae (Gerber and Stocker, 2007; Widmann et al., 2017). These approaches all require the larvae to be transferred manually several times from one Petri dish to another during the procedure and are thus labor intensive. To overcome this limitation, we aimed to develop a new, robust, and easy-to-handle device, which we named Maggot Instructor, to train Drosophila larvae in an automated fashion. The device consists of a training box connected to a computer that controls the type and timing of the applied stimuli via a cRIO system and an ATD (**Figures 1A,D**) (Dylla et al., 2017). Both are programmed by simple and flexible customizable training protocols using a BDOS software (Dylla et al., 2017). The training box consists of 10 separate training chambers that can be regulated in parallel or individually (**Figures 1A,B**). Therefore, one can train up to 10 groups of larvae in this device in parallel to increase the throughput. Each training chamber consists of a case, in which an electrode is incorporated at the front and the rear end (**Figure 1C**, above). In addition, a Peltier element is placed underneath the chamber and the odor inlets and outlets on all four sides (**Figure 1C**, above). The training chamber is closed at the top by a lid equipped with a white and a blue LED (**Figure 1C**, below). Therefore, the larvae can be exposed to the following stimuli: cold, heat, air, electric shock, and light (white and blue). Additional technical details are included in **Figure 1**, in Section "Materials and Methods," or are available upon request. Our initial study focused on a protocol that automatically conditions the larvae by optogenetic activation of ORNs (CS) via blue light and stimulation through electric shock (US).

#### Training Procedure

As shown in several studies, Drosophila larvae can establish an aversive olfactory memory by associating an odor with an electric shock (Aceves-Pina and Quinn, 1979; Heisenberg et al., 1985; Tully et al., 1994; Khurana et al., 2009; Pauls et al., 2010a). The current model suggests that the olfactory information is signaled from ORNs via PNs to MB KCs (Ramaekers et al., 2005). MB KCs, which are third-order olfactory neurons, are also stimulated via DANs, which signal a negative reinforcement (Selcho et al., 2009). When both stimuli coincide, synaptic plasticity occurs. These changes imply that in the following test, MBONs can be addressed by the learned odor to trigger the learned behavior (**Figures 2A,B**). In the standard assays, odors are used as CS. However, extensive preliminary tests have shown that using odors lead to different problems, including sticking to the agarose substrate in the training chamber (data not shown). Agarose is required to provide a substrate on which larvae can crawl easily and to prevent the larvae from drying out (Apostolopoulou et al., 2014). For this reason, we decided to train the larvae not with real odors but through the optogenetic activation of individual ORNs. Honda et al. (2014) have shown that the artificial optogenetic activation of a single ORN is sufficient to induce an associative olfactory memory in Drosophila larvae.

The two-odor reciprocal training paradigm is a widely used method to study associative olfactory learning and memory in larvae (Aceves-Pina and Quinn, 1979; Gerber and Stocker, 2007; Schipanski et al., 2008; Eschbach et al., 2011; von Essen et al., 2011; El-Keredy et al., 2012; Widmann et al., 2017). The use of a similar design would therefore allow for the comparison of larval odor-taste and odor-electric shock learning and memory in general. However, in an early study, we have shown that this design features several caveats (Pauls et al., 2010a). (i) The method yields relative low performance scores and thus may cause difficulty in the comparative studies of genetically manipulated larvae. (ii) This drawback may be partially overcome by increasing the number of training cycles but trigger starvationdependent effects. (iii) The two-odor design causes a sequence effect as differences are observed in the performance depending on whether the first (CS1) or second odor (CS2) has been punished. To overcome these concerns, we decided to use exactly the same one-odor non-reciprocal training design parameters, which we have established in our previous work (Pauls et al., 2010a).

The automated training protocol consists of a 60 s blue light phase, in which an electric shock is applied during the last 30 s, followed by a 300 s resting phase in complete darkness (**Figures 2C,E**). The training trial is repeated 10 times (from now on called 10-cycle training). Immediately thereafter, the larvae

Input Output), maggot stimulator and a training box. The training box is split in ten training chambers to parallelize larval training. Each training chamber has a source of light and electric shock. (B) Shows the training box on top and its cover at the bottom. (C) Shows a training chamber at the top and its lid that includes two LEDs at the bottom. (D) Shows the compactRIO system and the connected custom-made automated training device. b, w, v, and t show the connections for the blue and white light, the voltage channel and the temperature channel, respectively.

are tested for 5 min for their odor preference for a specific odor over paraffin oil, which serves as the control (**Figure 2D**). The test therefore requires a manual step.

## Pairing Optogenetic Or47a Activation With Electric Shock Reduces Larval Preferences for Amyl Acetate

To demonstrate that Drosophila larvae can be trained in an automated fashion via the Maggot Instructor, different parameters had to be tested in advance. We used the artificial blue-light dependent activation of Or42b-Gal4 and Or47a-Gal4 crossed with UAS-ChR2-XXL to specifically activate ORN 42b and 47a, respectively (Dawydow et al., 2014; Honda et al., 2014). Both lines were reported to be single-cell-specific (Fishilevich et al., 2005). ORN 47a was reported to specifically encode the odor amyl acetate (AM), whereas ORN 42b encodes the odor ethyl acetate (EA) (Kreher et al., 2005; Hoare et al., 2011).

We initially focused our analysis on ORN 47a and checked whether the larvae that express ChR2-XXL in ORN 47a can

FIGURE 3 | Naïve olfactory choice for amyl acetate and benzaldehyde. (A) Schematic representation of naïve olfactory choice for amyl acetate. Olfactory perception is analyzed by putting about 30 larvae in the middle of a Petri dish with an amyl acetate containing odor container (AM, red) on one side and an paraffin oil containing container (CC, turquoise) on the other side. After 5 min larvae are counted to calculate an olfactory preference index. (B) Schematic representation of naïve olfactory choice for amyl acetate. Olfactory perception is analyzed by putting 30 larvae in the middle of a Petri dish with a benzaldehyde containing odor container (BA, green) on one side and an empty container (CC, turquoise) on the other side. After 5 min larvae are counted to calculate an olfactory preference index. (C) The behavioral response for amyl acetate (1:500 dilution) of Or47a-Gal4/UAS-ChR2-XXL, Or47a-Gal4/+ and UAS-ChR2-XXL/+ larvae were statistically not significant from each other (Kruskal–Wallis, p = 0.0.118). All three groups showed an olfactory preference index statistically significantly different from zero (one sample t-test, p < 0.0001, for all three groups). (D) The behavioral response for benzaldehyde (undiluted) in Or47a-Gal4/UAS-ChR2-XXL, Or47a-Gal4/+ and UAS-ChR2-XXL/+ larvae were statistically not significant from each other (one-way ANOVA, p = 0.5757). All three groups showed an olfactory preference index statistically significantly different from zero (one sample t-test, p = 0.0196, p = 0.0012, p < 0.0001, respectively). Differences between groups are depicted below the respective box plots, at which ns indicates p ≥ 0.05. Small circles indicate outliers. Sample size is indicated with the letter n.

perceive odors. The larvae were tested for their naïve olfactory choice behavior between an odor-filled container on one side and a container without olfactory cue on the other side of a Petri dish (**Figure 3**). This test was performed with either AM or benzaldehyde (BA) as odor stimuli (**Figures 3A,B**). Or47a-Gal4/UAS-ChR2-XXL larvae are attracted by the odor AM (**Figure 3C**). This behavioral response shows no significant difference from both the control groups (Or47-Gal4/+ and UAS-ChR2-XXL/+) (**Figure 3C**). Similarly, BA is attractive to Or47a-Gal4/UAS-ChR2-XXL larvae, and the response is comparable in both control groups (Or47-Gal4/+ and UAS-ChR2-XXL/+) (**Figure 3D**). We concluded that the expression of ChR2-XXL in ORN 47a exerts no influence on the naïve odor perception of the larvae.

We then tested whether the activation of ORN 47a, together with an electric shock leads to a reduction in the odor preference for AM (**Figure 4**). This reduction would indicate that an aversive olfactory memory was formed. We performed five different experiments in which the light intensity and the voltage of the electric shock remained unchanged during training, but the dilution of AM in paraffin oil in the test was either 1:100, 1:250, 1:500, 1:600, or 1:750 (**Figures 4B–F**). During training via the Maggot Instructor, all larvae received the 10-cycle training as described before (**Figures 2E**, **4A**). As a result, we observed that for the dilutions 1:100, 1:250, and 1:500, the Or47a-Gal4/UAS-ChR2-XXL larvae showed a reduced olfactory preference for AM compared with both genetic control groups (Or47a-Gal4/+ and UAS-ChR2-XXL/+) (**Figures 4B–D**). No difference was observed between the three groups when the dilution of AM was 1:600 or 1:750 in the test (**Figures 4E,F**). These results suggest that associative olfactory conditioning using the Maggot Instructor is feasible, and Drosophila larvae are very likely able to establish an aversive odor-electric shock memory. However, the memory can only be revealed at high odor concentrations. The olfactory preference for AM for both the control groups (Or47a-Gal4/+ and UAS-ChR2-XXL/+) statistically significantly differed from each other when a dilution of 1:500 was used (**Figure 4D**). Nevertheless, we decided to continually use this odor dilution as the experimental group (Or47a-Gal4/UAS-ChR2-XXL) features a specific behavioral phenotype in comparison with both the control groups (Or47a-Gal4/+ and UAS-ChR2-XXL/+), and we have used the lowest possible odor concentration to avoid the harmful side effects.

# The Performance After Maggot Instructor Training Depends on the Applied Electric Shock and Light Intensities

Next, we performed a parametric analysis with varying voltage of the applied electric shock and intensity of the artificial blue light activation (**Figures 5**, **6**). We used the established 1:500 AM dilution and the 10-cycle protocol (**Figure 5A**) and tested whether electric shocks applied at 60, 90, or 120 V cause different effects on learning and memory (**Figures 5B–D**). As a result, we noted that for electric shocks of 60 and 120 V, in contrast to 90 V, Or47a-Gal4/UAS-ChR2-XXL larvae showed a reduced olfactory preference for AM compared with both the genetic control groups (Or47a-Gal4/+ and UAS-ChR2-XXL/+) (**Figures 5B–D**). Based on this results, we continually used 120 V for electric shocks, as all larvae survived this treatment and

(Continued)

#### FIGURE 4 | Continued

fpsyg-09-01010 June 18, 2018 Time: 16:9 # 9

an olfactory preference for amyl acetate statistically significant from zero (one sample t-test, p < 0.0001 for all three groups). (C) The expression of ChR2-XXL in ORN 47a led to a reduction of olfactory preference for amyl acetate at a dilution of 1:250 (Dunn's multiple pairwise comparison, p = 0.0.0035, p = 0.0307, respectively). All three groups showed olfactory preferences for amyl acetate statistically significant from zero (one sample t-test, p < 0.0001 for all three groups). (D) The expression of ChR2-XXL in ORN 47a led to a reduction of olfactory preference for amyl acetate at a dilution of 1:500 (one-way ANOVA, p < 0.0001). However, both control groups (Or47-Gal4/+ and UAS-ChR2-XXL/+) exhibited olfactory preferences, which are statistically significant form each other (Tukey post hoc test, p = 0.0001). All three groups showed an olfactory preference for amyl acetate statistically significant from zero (one sample t-test, p < 0.0001 for all three groups). (E) All three groups showed olfactory preferences for amyl acetate at a dilution of 1:600, which are statistically significant from zero (one sample t-test, p < 0.0001 for all three groups) but statistically not significant from each other (one-way ANOVA, p = 0.057). (F) All three groups showed olfactory preferences for amyl acetate at a dilution of 1:750, which are statistically significant from zero (one sample t-test, p = 0.0002, p < 0.0001, p = 0.0002, respectively) but statistically not significant from each other (one-way ANOVA, p = 0.0746). Differences between groups are depicted below the respective box plots, at which ns indicates p ≥ 0.05. Different lowercase letters indicate statistical significant differences at level p < 0.05. Small circles indicate outliers. Sample size is indicated with the letter n.

FIGURE 5 | Odor-electric shock learning and memory in Drosophila larvae depends on the applied voltage of the electric shock. (A) Timescale of associative conditioning using 10 cycles, different voltages for electric shocks (60, 90, and 120 V) and continuous blue light with an intensity of 100%. For the olfactory preference test amyl acetate with a dilution of 1:500 was used. (B) Using 60 V in the training procedure led to a reduction of olfactory preferences for Or47-Gal4/UAS-ChR2-XXL larvae compared to both genetic controls (Or47-Gal4/+ and UAS-ChR2-XXL/+) (Tukey post hoc test, p = 0.001, p = 0.0168, respectively). Both genetic controls showed olfactory preferences, which are statically significant from zero (one sample t-test, p < 0.0001 for both groups), whereas Or47-Gal4/UAS-ChR2-XXL larvae showed an olfactory preference, which is not statistically significant from zero (one sample t-test, p = 0.068). (C) Using 90 V in the training procedure led to a reduction of olfactory preferences for all three groups, which are statistically not significant from each other (one-way ANOVA, p = 0.5917). All three groups showed olfactory preferences, which are statically significant from zero (one sample t-test, p = 0.0375, p = 0.0004, p = 0.0025, respectively). (D) The olfactory preference for amyl acetate conditioned with 120 V was already analyzed in Figure 4D and is just shown for comparison. Differences between groups are depicted below the respective box plots, at which ns indicates p ≥ 0.05. Different lowercase letters indicate statistically significant differences at level p < 0.05. Small circles indicate outliers. Sample size is indicated with the letter n.

FIGURE 6 | Odor-electric shock learning and memory in Drosophila larvae is dependent on the intensity of the blue light. (A) Timescale of associative conditioning using 10 cycles, 120 V for electric shocks and continuous blue light with different intensities (50, 75, and 100%). For the olfactory preference test amyl acetate with a dilution of 1:500 was used. (B) Using a light intensity of 50% in the training procedure led to olfactory preferences, which are statistically significant within the three groups (one-way ANOVA, p = 0.0288). However, the difference was only statistically significant between Or47-Gal4/UAS-ChR2-XXL and Or47-Gal4/+ larvae (Tukey post hoc test, p = 0.0222), whereas the olfactory preferences for Or47-Gal4/UAS-ChR2-XXL and UAS-ChR2-XXL/+ larvae were not statistically significant from each other (Tukey post hoc test, p = 0.2906). (C) Using a light intensity of 75% in the training procedure led to olfactory preferences, which are statistically not significant from each other (one-way ANOVA, p = 0.0522). All three groups showed olfactory preferences, which are statically significant from zero (one sample t-test, p < 0.0001, p < 0.0001, p = 0.007, respectively). (D) The olfactory preference for amyl acetate conditioned with 120 V was already analyzed in Figure 4D and is just shown for comparison. Differences between groups are depicted below the respective box plots, at which ns indicates p ≥ 0.05. Different lowercase letters indicate statistically significant differences at level p < 0.05. Small circles indicate outliers. Sample size is indicated with the letter n.

showed slightly stronger differences between the experimental group and both controls.

Next, we used the 1:500 AM dilution, 10-cycle, and 120 V protocol (**Figure 6A**) to test whether three different blue light intensities (50%, 75%, or 100%) cause different effects on learning and memory (**Figures 6B–D**). We noted that for blue light intensities of 100%, Or47a-Gal4/UAS-ChR2-XXL experimental larvae showed a reduced olfactory preference for AM compared with both genetic controls (Or47a-Gal4/+ and UAS-ChR2-XXL/+) (**Figure 6D**). By contrast, when trained with blue light intensities of 50% and 75%, the Or47a-Gal4/UAS-ChR2-XXL larvae showed no significant reduction in their preference for AM compared with both or at least one genetic control (**Figures 6B,C**; for blue light intensities of 50%, a significant difference was observed between Or47a-Gal4/+ and Or47a-Gal4/ChR2-XXL). Based on this

result, we used a blue light intensity of 100% for follow-up experiments.

# Lithium Chloride Application or Pulsed Blue-Light Causes no Improvement in the Training Protocol

Previous studies that used LiCl reported an increase in larval memory scores for odor-electric shock learning as it makes the agarose substrate electrically conductive while being tasteless for larvae (Aceves-Pina and Quinn, 1979). However, this effect could not be confirmed by a study from our laboratory (Pauls et al., 2010a). Nonetheless, we determined whether the use of LiCl affects the automated Maggot Instructor training as its intake might cause harmful effects for the larvae and was reported to modulate adult behavior (Ries et al., 2017). The obtained data revealed that the use of LiCl is not necessary in our setup (**Figure 7B**), similar to our published data (Pauls et al., 2010a).

Prolonged blue-light activation of the sensory neurons via ChR2-XXL can lead to a decrease in firing of the cells (Dawydow et al., 2014). Therefore, we tested whether pulsed blue light activation of ORN 47a may produce a stronger behavioral effect. Instead, of a constant blue light activation of 60 s we used an alternating 1 s on-off regime. In this case, Or47a-Gal4/UAS-ChR2-XXL experimental larvae showed a significant reduction in their odor preference compared with the Or47a-Gal4/+ and UAS-ChR2-XXL/+ control groups (**Figure 7C**). Direct comparison of the performance of Or47a-Gal4/UAS-ChR2-XXL larvae at pulsed light (**Figure 7C**) and constant light (**Figure 7D**) showed a significant difference in the odor preference between both groups. This result indicates that the optogenetic activation with pulsed light featured a weaker effect on reducing odor preferences for AM than with constant light. Therefore, we continually used the 1:500 AM dilution, 10-cycle, 120 V, and 100% constant blue light protocol on the agarose filled training chambers without LiCl.

# Additional Control Experiments Support the Associative Nature of the Learning and Memory Phenotype

The conditioning regime used by the Maggot Instructor lacks reciprocity. The regime defines learning and memory as a reduction in AM preference between an experimental

conditioning using 10 cycles, continuous blue light with an intensity of 100%, without electric shock. For the olfactory preference test amyl acetate with a dilution of 1:500 was used. (B) Timescale of associative conditioning using 10 cycles, 120 V for electric shocks, but without continuous blue light. For the olfactory preference test amyl acetate with a dilution of 1:500 was used. (C) Associative conditioning without electric shock stimulation but optogenetic Or47a activation led to olfactory preferences, which are statistically not significant within the three groups (one-way ANOVA, p = 0.4062). All three groups showed olfactory preferences, which are statistically significant from zero (one sample t-test, p < 0.0001 for all three groups). (D) Associative conditioning without optogenetic Or47a activation but electric shock stimulation led to olfactory preferences, which are statistically not significant within the three groups (one-way ANOVA, p = 0.3355). All three groups showed olfactory preferences, which are statistically significant from zero (one sample t-test, p < 0.0001 for all three groups). (E) The olfactory preference for amyl acetate conditioned with 120 V was already analyzed in Figure 4D and is just shown for comparison. Differences between groups are depicted below the respective box plots, at which ns indicates p ≥ 0.05. Different lowercase letters indicate statistical significant differences at level p < 0.05. Small circles indicate outliers. Sample size is indicated with the letter n.

group and two genetic control groups. We thus designed two additional control experiments to ensure that neither blue light activation nor electric shock stimulation alone specifically can change the AM preference of Or47a-Gal4/UAS-ChR2- XXL larvae (**Figures 8A,B**). Although unlikely, significant differences between the experimental and control groups would suggest that the obtained phenotype would be based on non-associative effects rather than associative learning and memory. As expected, both results showed no reduction in the AM preference of the Or47a-Gal4/UAS-ChR2-XXL larvae compared with the Or47a-Gal4/+ and UAS-ChR2-XXL/+ control groups (**Figures 8C,D**). These results show that the observed behavioral change in the experimental larvae after conditioning via the Maggot Instructor is based on associative learning and memory.

# Artificial Activation of Distinct ORNs Establishes Odor-Specific Memories

Next, we analyzed the odor specificity of the memory. Studies previously showed that artificial activation of a ORN during conditioning induces an odor-specific memory that overlaps with the response profile predicted for the respective ORN (Honda et al., 2014). Accordingly, we tested whether the artificial activation of ORN 47a can also establish odor-electric shock learning and memory for an odor that is not covered by the reported Or47a response profile. Considering Or47a, such case applies to BA (Kreher et al., 2005; Hoare et al., 2011; Munch and Galizia, 2016). As expected Or47a-Gal4/UAS-ChR2-XXL larvae showed an odor preference for BA, and this preference is indistinguishable from the both genetic control groups (Or47a-Gal4/+ and UAS-ChR2-XXL/+) (**Figure 9B**). Based on this result we conclude that odor-electric shock learning and memory established after training via the Maggot Instructor is specific for the activated ORN and thus overlaps with its reported response profile. We confirmed this result independently by reproducing the finding published for Or42b. Honda et al. (2014) reported that the artificial activation of ORN 42b paired with an artificial activation of octopaminergic neurons that encode for a rewarding function establishes an appetitive olfactory memory specific for EA. Using our standardized training protocol but the odor EA (1:1000) in the test (**Figure 10A**) Or42b-Gal4/UAS-ChR2-XXL larvae also established an aversive odor-electric shock memory (**Figure 10B**).

#### DISCUSSION

## The Maggot Instructor Trains Larvae in an Automated Fashion to Establish an Associative Olfactory Memory

Drosophila larvae can establish different types of associative memory based on the pairing of two stimuli (US and CS) (Aceves-Pina and Quinn, 1979; Scherer et al., 2003; Gerber and Stocker, 2007; Widmann et al., 2017). In contrast to the almost exclusively manual assays that are currently in use, we showed that larvae can also be trained automatically with the

help of the Maggot Instructor. Automation will allow one to conduct comprehensive behavioral screens of newly established genetic tools (Li et al., 2014; Saumweber et al., 2018). In several experiments, we have shown that genetically modified larvae, which still show a natural naïve odor preference (**Figure 3**), learn the temporal paired optogenetic activation of ORN 47a with an electric shock and store this experience as an aversive olfactory memory (**Figures 4**–**7**, **10**). Our results showed that

this memory is specific for the identity and concentration of odors as the odor-electric shock memory was only detectable at certain concentrations of AM (**Figure 4**) and not visible when BA was used in the test (**Figure 9**). The conclusion regarding the associative nature of the observed reduction in the AM preference is compelling as we also showed that other parameters per se, such as artificial activation and electric shock, caused no alteration in the tested olfactory behavior (**Figure 8**). Therefore, we conclude that training larvae via the Maggot Instructor leads to an odor-specific associative process. The formation of memory by artificial activation of ORNs is not limited to ORN 47a given that an EA memory can be formed through the activation of ORNS 42b (**Figure 10**). However, for each of the 21 ORNs, odor-specific associative processes have to be tested, as several studies have shown the presence of non-equivalency among larval ORNs (Mathew et al., 2013; Hernandez-Nunez et al., 2015; Newquist et al., 2016). ORN 42a, for instance, unlike many other larval ORNs was shown to respond to a wide range of odors (Kreher et al., 2005; Hoare et al., 2011; Mathew et al., 2013).

# Real World Stimulation or Artificial Activation of Distinct Neurons of the Learning and Memory Network

To establish an associative olfactory memory in Drosophila larvae, the animals with natural stimuli, such as an odor and an electric shock, must be conditioned (Aceves-Pina and Quinn, 1979; Pauls et al., 2010a). However, the precise control of natural stimuli often presents difficulty. Therefore, thermogenetic and optogenetic effectors, such as TRPA1 and ChR2, that are expressed via transgenic techniques provide an alternative as they allow for the precise control of the activity of defined neurons in living larvae (Hamada et al., 2008; Dawydow et al., 2014). Associative olfactory conditioning theoretically includes the CS (odor) and/or the US (reward/punishment) pathways. Schroll et al. (2006) showed that light-induced activation of a set of DA neurons paired with an odor stimulus induces aversive memory formation, whereas activation of OA neurons induces appetitive memory formation. These results could be extended by demonstrating that in downstream of the OA neurons, the activity of four DA pPAM is also sufficient to trigger an appetitive memory (Rohwedder et al., 2016). For two of these DA neurons, activating them individually is enough for memory formation (Saumweber et al., 2018). In summary, these studies showed that substation experiments can be possibly carried out for the US in the larva, both for appetitive and for aversive learning, up to the single-cell level. This condition also holds true for the adult Drosophila. By contrast, a successful CS substitution at the level of ORN has thus far only been shown for the larva stage (Honda et al., 2014). Perhaps, the reason is the simpler neural network or the organization of parts of the larval olfactory pathway as a labeled line up to the MB (Ramaekers et al., 2005; Berck et al., 2016; Eichler et al., 2017). The optogenetic activation of ORN 24a and ORN 42b paired with the thermogenetic activation of most OA neurons induces an appetitive memory for acetophenone and EA, respectively (Honda et al., 2014). In this study, we showed for the first time the establishment of an aversive memory via CS substitution (**Figure 10**). Taken together the activation of ORN 42b serves the classical CS function. The pairing of ORN 42b activation via a natural odor or artificially via blue light and a reward or punishment causes the CS to trigger attraction or avoidance. As a consequence, appetitive and

aversive associative learning processes can now be generated artificially, temporally, and spatially in various combinations in the larval brain and independent of natural stimuli. In this situation, the Maggot Instructor can be helpful. Thus, in future experiments, the order of CS and US, their precise timing (e.g., backward and forward conditioning; delay conditioning), and additional parameters, such as the number of training cycles or the strength of the CS and the US, can be analyzed in a controlled manner. The same condition applies to the neuronal networks. Activation experiments for PNs, sets of KCs, MBONs, and screens for identifying neurons of the US pathway would be conceivable.

#### Meaning of the Artificial ORN Activation

The associative olfactory learning and memory that we tested with ORN 47a was specific for AM (**Figures 4–9**). However, we opted not to analyze in-depth the odor specificity of the memory. The tuning curve for the receptor Or47a is very specific at low odor concentrations (10−<sup>4</sup> ) and responded almost exclusively to AM when tested for 26 different odors (Kreher et al., 2008). This result was also confirmed by a second study, which has tested for 19 different odors (Hoare et al., 2011). We used these results to select Or47a for our experiments. At a higher concentration (10−<sup>2</sup> ), the receptor specificity changes, and in addition to AM, one also sees responses to other odors, such as propyl acetate, isoamyl acetate, 1-octen-3-ol, and 2 heptanone. For the receptor Or42b, this condition is very similar. At low concentrations (10−<sup>4</sup> ) Or42b shows high specificity for EA. At high concentrations (10−<sup>2</sup> ) responses for ethyl butyrate, propyl acetate, 2,3-butanedione and potential AM are reported (Kreher et al., 2008; Hoare et al., 2011). The high throughput rate of the Maggot Instructor allows repetition of these physiological experiments at the behavioral level to identify the tuning curves for each ORN in relation to many odors after olfactory learning and memory. These experiments would provide more information on the neural principles of larval odor processing to better understand the odors that larvae can learn and remember.

#### Technical Caveats

The Maggot Instructor shortens the time necessary to perform an experiment. The manual training protocol consists of 60 s CS and US pairing followed by a 300 s resting phase in complete darkness (**Figures 2C,E**). This training trial is repeated 10 times and spans 60 min in total (Pauls et al., 2010a). Although the Maggot Instructor, compared with the manual protocol, requires about the same time to prepare the larvae before and test them after training, the training itself requires no handling. A standard experiment usually consists of an experimental group, a driver and reporter control, each with about 10 repetitions per genotype. This situation results in a time of approximately 3 (genotypes) × 10 (repetitions) × 60 min, or 30 h saved per complete experiment.

Although this rough estimate shows the immense time saved, one must also mention that large genetic screens cannot be achieved immediately. The Maggot Instructor requires ChR2 to be expressed in individual ORNs. This goal can be achieved either via direct Or promoter ChR2 fusion constructs, via the LexA, the Q, or the Gal4/UAS system (Brand and Perrimon, 1993; Lai and Lee, 2006; Potter et al., 2010). However, as these tools are either non-existent, rare, or problematic, and as they affect other genetic modifications, genetic screens require a special strategy to deploy the Maggot Instructor. For example establishing the Or47::ChR2- XXL larvae would be possible. This construct can either be combined with a MB-Gal4 line to screen for the requirement of individual genes using available UAS-RNAi lines or with UASshits to use available Gal4 and split-Gal4 lines to identify the neuronal circuits and individual neurons required for learning and memory (Kitamoto, 2001; Pfeiffer et al., 2010; Li et al., 2014). Alternatively, one can combine Or-LexA with LexAop-ChR2- XXL to artificially activate individual ORNs (Selcho et al., 2017). However, to date, to our knowledge, only Or47b-LexA (Hueston et al., 2016), which is not expressed in the larval olfactory system, has been published; thus, one would have to establish in any case new genetic tools before one can use the Maggot Instructor for large genetic screens.

#### Outlook

In this work, we exclusively focused on the aversive olfactory memory reinforced with electric shock. The design of the Maggot Instructor, however, allows a whole series of other applications. Drosophila larvae can also associate odor information with light or heat punishment (von Essen et al., 2011; Khurana et al., 2012). The Maggot Instructor can apply these stimuli automatically. Furthermore, the Maggot Instructor offers the possibility to analyze associative visual learning and memory by pairing a light stimulus with electric shock. Such a protocol is already established as a manual assay (von Essen et al., 2011).

Extensive double activation experiments are also now possible. Defined ORN activation (standardized CS) can then be paired with activation of individual sensory neurons expressing gustatory receptors, ionotropic receptors, transient receptor potential cation channels, and/or pickpocket ion channel genes (Clyne, 2000; Dunipace et al., 2001; Scott et al., 2001; Liu et al., 2003; Montell, 2005; Benton et al., 2009). In this manner, one could comprehensively identify the sensory neurons that encode for appetitive and aversive reinforcement in Drosophila larvae (e.g., Gr93a for aversive reinforcement and IR60c potentially for appetitive reinforcement) (Apostolopoulou et al., 2016; Croset et al., 2016).

In summary, the range of applications of the Maggot Instructors extends well beyond the one shown here. Therefore, we confidently present in this work a very useful device that allows more rapid analysis of the behavioral, neuronal, and molecular fundamentals and different forms of larval learning and memory in the future.

#### AUTHOR CONTRIBUTIONS

UT conceived the study, coordinated and contributed behavioral experiments, analyzed behavioral data, designed figures, and wrote the manuscript. AW conceived the study, coordinated

behavioral experiments, analyzed behavioral data, designed figures, and wrote the manuscript. AT conceived of and coordinated study, analyzed the data, designed figures, and wrote the manuscript.

#### FUNDING

This work was supported by the DFG grants (TH1584/1-1 and TH1584/3-1) and the Zukunftskolleg of the University of Konstanz (all to AT).

#### REFERENCES


#### ACKNOWLEDGMENTS

We thank Dr. Tilman Triphan, Dr. Wolf Hütteroth, Dr. Astrid Rohwedder, and Dr. Dennis Pauls for their fruitful comments on the manuscript. Additionally, we thank Lyubov Pankevych and Margarete Ehrenfried for fly care and maintenance. In addition, we want to express our special thanks to the workshop of the University of Konstanz for constructing the Maggot Instructor. We also thank Johanna Wörner for her help in establishing the device. Special thanks also go to Prof. Brian Smith for his help and advice.




**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2018 Tomasiunaite, Widmann and Thum. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# The Drivers of Heuristic Optimization in Insect Object Manufacture and Use

#### Natasha Mhatre<sup>1</sup> \* and Daniel Robert<sup>2</sup>

<sup>1</sup> Department of Biological Sciences, University of Toronto at Scarborough, Scarborough, ON, Canada, <sup>2</sup> School of Biological Sciences, University of Bristol, Bristol, United Kingdom

Insects have small brains and heuristics or 'rules of thumb' are proposed here to be a good model for how insects optimize the objects they make and use. Generally, heuristics are thought to increase the speed of decision making by reducing the computational resources needed for making decisions. By corollary, heuristic decisions are also deemed to impose a compromise in decision accuracy. Using examples from object optimization behavior in insects, we will argue that heuristics do not inevitably imply a lower computational burden or lower decision accuracy. We also show that heuristic optimization may be driven by certain features of the optimization problem itself: the properties of the object being optimized, the biology of the insect, and the properties of the function being optimized. We also delineate the structural conditions under which heuristic optimization may achieve accuracy equivalent to or better than more fine-grained and onerous optimization methods.

#### Edited by:

Martin Giurfa, UMR5169 Centre de Recherches sur la Cognition Animale (CRCA), France

#### Reviewed by:

Aurore Avargues-Weber, UMR5169 Centre de Recherches sur la Cognition Animale (CRCA), France Cédric Sueur, UMR7178 Institut Pluridisciplinaire Hubert Curien (IPHC), France

#### \*Correspondence:

Natasha Mhatre natasha.mhatre@gmail.com

#### Specialty section:

This article was submitted to Comparative Psychology, a section of the journal Frontiers in Psychology

Received: 01 March 2018 Accepted: 31 May 2018 Published: 21 June 2018

#### Citation:

Mhatre N and Robert D (2018) The Drivers of Heuristic Optimization in Insect Object Manufacture and Use. Front. Psychol. 9:1015. doi: 10.3389/fpsyg.2018.01015 Keywords: insect cognition, object manufacture, baffling behavior, objective function, optimization, heuristics

"It is demonstrable," said he, "that things cannot be otherwise than they are; for as all things have been created for some end, they must necessarily be created for the best end."

Candide, or Optimism – Voltaire

# OBJECT MANUFACTURE BY INSECTS

Animals make and use a large variety of objects for a range of functions, mainly constructions that they inhabit or use as traps, tools that they use for food acquisition or for increasing their reach, or objects they use to create displays that attract mates or warn rivals (Hansell, 2007). Interestingly, insects seem to participate in the full gamut of object use and manufacture despite their small body- and brain size. Indeed, it is likely that object manufacture is more prevalent in insects than in non-human vertebrates (Hansell, 2007). For instance, insects make a variety of intricate nests and inhabitations that provide protection and even climate control (Korb, 2007). Ant-lions build pitfall traps to capture ground dwelling prey (Devetak et al., 2005; Fertin and Casas, 2006), ants drop stones and soil using them as projectile weapons as they raid a bee's nest for pollen (Lin, 1964; Schultz, 1982). Other insects like caddis-fly, lacewing and reduviid larvae defend or camouflage themselves by covering themselves with debris (Livingstone and Ambrose, 1986; Ferry et al., 2013; Tauber and Tauber, 2014). Crickets manufacture acoustic objects like baffles and burrows that help them increase the loudness of their mating calls (Bennet-Clark, 1987; Mhatre et al., 2017).

More uniquely, insects are the only animals known to make objects out of their own bodies, objects that have been dubbed self-assemblages (Anderson et al., 2002). For example, ants make bridges that help them travel more efficiently over gaps in the substrate (Reid et al., 2015; Graham et al., 2017), others make rafts that allow them to float and survive a flood (Mlot et al., 2011). Bees are even known to make predator killing ovens using their own body heat (Ken et al., 2005).

Many of these objects have to be made in variable environmental contexts without fixed properties. Thus, optimal or even functional manufacture may not be possible using a stereotyped behavioral program. It is proposed that manufacture behavior needs to be responsive to features of the environmental demands that constitute the problem. For instance, some objects must function in different weather: termite mound architecture has to respond to local climatic conditions. In West Africa, shrub savanna conditions are warm but thermally unstable, and termite mounds with many ridges and turrets function better, whereas compact mounds with a dome-like structure perform better in the cooler but more stable gallery forests (Korb and Linsenmair, 1998). Thus, it can be hypothesized that a behavioral program that flexibly responds to local climatic conditions that the termites find themselves in, would be better than a stereotypical, "one size fits all" behavior. Another problem feature that calls for flexibility is material efficiency: the range of materials that can be used to manufacture the objects, each of which may have different efficiencies, requires animals to be able to select intelligently within the range available to them. Material efficiency is understood here as the collection of material properties that confer functionality in specific uses, e.g., hardness, size, weight, color, insulation, biodegradability, wettability, thermal mass, to name a few. For example, tree crickets can choose from a wider variety of leaf sizes to make baffles, and this distribution changes as the seasons and hence plant phenology progresses. However, only a narrow range of the available leaves make an optimal or 'worthwhile' baffle, which the crickets must choose from while also balancing search times (Mhatre et al., 2017). In still other cases, the problem itself may be variable and may necessitate a flexible behavioral program. For example, the size of the gap in the environment over which army ants must build a bridge is dependent on the environment itself, and hence highly variable. The ants must follow a behavioral program that enables flexible decision making that balances the cost of ants used to make the bridge against the travel distance saved (Reid et al., 2015; Graham et al., 2017). In view of this complexity, establishing whether and how optimal solutions are reached in insects remains a challenging task.

It was commonly believed that insects have simple and highly stereotypical behavior as a result of a small brain size. Insect behavior was proposed to follow stereotypical and rather inflexible behaviors, the so-called fixed action patterns, in response to a particular set of sensory inputs (Gould and Gould, 1982). This description of insect behavior is at odds with the evidence and need for flexibility in object manufacture described in more recent literature. Indeed, it is also at odds with the proposition that insects optimize the objects they manufacture. More recent work, in effect, has challenged the notion that larger brains are better at cognitive tasks (Chittka and Niven, 2009). Small brained insects have been demonstrated to exhibit remarkably sophisticated cognitive behavior, such as numerosity (Dacke and Srinivasan, 2008) and concept abstraction (Giurfa et al., 2001; Avarguès-Weber et al., 2012). Using a 'constructive' approach, researchers have found that even relatively small neural networks can produce similarly sophisticated behavior (Dehaene et al., 1987; Dehaene and Changeux, 1993; Beer, 2003). The idea that even minimal computational programs can enable flexible, responsive and hence intelligent behavior is increasingly gaining traction. In this review, we consider the different kinds of minimal computational procedures that might be used by insects to optimize the objects they manufacture.

# OPTIMIZATION: DEFINITIONS AND CRITICISMS

In formal mathematical terms, optimization is a process by which we find the maximum or minimum value of some function (**Figure 1C**). This function is called the objective function, and

FIGURE 1 | A baffle as an example of an object optimization problem. (A) Tree cricket males cut a hole in a leaf to create an object called a baffle. (B) This baffle is used by the cricket during singing and the male places its wings against the hole and parallel to the leaf surface while producing sound from its vibrating wings. (C) This device improves the crickets sound radiation efficiency. This efficiency is controlled by two main parameters, the size of the leaf they use and the size of the hole (reproduced from data in Mhatre et al., 2017). To optimize a baffle, the animal must find the highest point on this objective function. This is a 2 dimensional objective function. Objective functions, in general, however, may be determined by any number of parameters and each parameter will correspond to one dimension of the function. A baffle can in principle have any combination of positive leaf length and hole length and this would form the domain of the problem. Realistically, however, the leaf sizes available to the insect depend on naturally available leaves (∼11–141 mm) and the hole sizes depend in turn on leaf size. This smaller subspace is called the feasible region of the objective function.

the input space over which this function varies is defined as the domain. Real world optimization problems tend to have a feasible domain, i.e., the space defined by the subset of inputs that can be realistically achieved (**Figure 1**). Such a simple definition, admittedly, hides the wide range of problem types and approaches encompassed in optimization theory (Foulds, 1948).

Optimization in a biological context poses different problems. It is generally understood that in most cases optimization behavior is inherited and the animal is not intentionally seeking the optimum of the objective function. It is expected that the animal has a preference for some quantity which it maximizes without intention. During this behavior, the animal does not know, or have an expectation of, how the objective function will change as it performs modifications to its behavior and it discovers this function via a search. Additionally, the suite of behaviors that are regarded to be under an optimization process can only arise through an evolutionary process, which itself involves interactions with the environment and other organisms. Unsurprisingly, as these interactions are dynamic and involve coevolutionary processes, such as arms-races, objective functions may be intrinsically dynamic and present varying optimal points (McFarland, 1977; Smith, 1978; Parker and Smith, 1990).

In this fluid context, the most serious challenge to the contention that animals can optimize is that it is an overly optimistic or 'Panglossian' outlook (Gould and Lewontin, 1979). Taking a page from Voltaire's play, Candide, Gould and Lewontin (1979) in their seminal paper argued against a then-prevalent tendency to view biological traits as unitary, and as having been optimized by evolutionary forces. They argued that it was more realistic to view traits as part of larger Baupläne and therefore constrained by phylogeny, development and physical and architectural constraints. The authors argued that this means that most traits are not likely to be functionally 'optimal' (Gould and Lewontin, 1979). Indeed, the idea that a biological trait develops for a single unambiguous function contains a teleological argument and is therefore problematic (Pierce and Ollason, 1987). Different authors have dealt with these criticisms. The chief response does not resolve the issues raised but argues instead that optimization can be a productive hypothesis; one that then enables insights into the cost-benefit trade-offs and the phylogenetic constraints inherent to each problem considered (Stearns and Schmid-Hempel, 1987; Parker and Smith, 1990; Alexander, 1996).

One kind of biological problem, however, seems to be exempt from some of these objections: object manufacture and use. Unlike biological traits, objects manufactured by animals serve one or a very small set of well-defined or definable functions (Hansell and Ruxton, 2008; Shumaker et al., 2011). Manufacturing objects commands effort. Some objects even require an investment for a reward that appears much later (Finn et al., 2009). The existence of investments made in reshaping external objects, which the animal can chose not to make, suggests that object manufacture behavior is functionally important. Additionally, most objects are made solely using external materials which are themselves not under natural or co-evolutionary selection and can be chosen by the user for their functional properties. This makes the problem significantly simpler by limiting natural selection to the cognitive processes and morphology that supports object manufacture and use. Therefore, it is possible that animals can evolve, and indeed inherit, behavioral processes or routines that optimize the objects they manufacture.

Finally, the problem of teleology, here the intentional 'goaldirected' behavior of the animal, does not apply to inherited optimization processes which arise through natural or sexual selection. In these cases, the animal does not need to 'intend' to optimize the object. A facile and hard to test claim made sometimes is that object manufacture and its optimization ought to involve cognitive 'insight' or 'innovation' on the part of the animal. Such position can be seen as tantamount to claiming that the animal demonstrates teleological or conscious goaldirected action. While such cognitive capacity cannot be excluded a priori, we defend the idea that it is neither parsimonious nor a necessity for explaining object manufacture, use and optimization in animals, including insects.

# HEURISTICS OR NOT?

What are the search processes for finding the optima of objective functions? For optimization problems where the objective function is not predefined or is large, there are three broad search methods: (1) those that stop after a set number of steps – finitely terminating searches, (2) convergent methods that search iteratively and stop when objective function value converges, and (3) heuristic methods which do neither, but rather provide a 'recipe'-like search method that is good at finding approximate solutions under certain circumstances.

A biological example of a finite terminating search in biology is a 'best-of-n' strategy for mate finding in which mate search is stopped after encounters with N males and the best male is chosen (Janetos, 1980; Dombrovsky and Perrin, 1994) or honey bees scouting a finite and set number of nest building sites and returning to the best of those sites (Seeley and Buhrman, 2001). In an object manufacture context, this would mean that the animal makes only N changes to the object and return to the object design that was the best among those N alternatives. An example of an iterative-convergent strategy would be the process by which mole crickets gradually improve the acoustic resonance of their singing burrow using sensory feedback (Bennet-Clark, 1987). In such a strategy, the mole cricket randomly changes different architectural features of the burrow, and continues only with those changes that make the burrow louder. Here, the important part is that the animal is monitoring the functional output rather than the architectural features of the structure it is building. An example of a heuristic search is baffle optimization in tree crickets which is guided by three rules that lead to an optimal baffle without a need for sensory feedback that monitors the cricket's loudness (Mhatre et al., 2017). Baffles, much like mole cricket burrows, are acoustic aids that allow tree crickets to increase the loudness of their mating call. To make a baffle, a tree cricket must make a hole in a leaf and must sing from within this hole. In effect, there are three important features to a baffle, leaf size, hole size, and hole position. Three rules are sufficient to acoustically

optimize the baffle, (1) pick the largest leaf, (2) make a hole the size of its wings, and (3) place the hole at the center of the leaf (Mhatre et al., 2017). Here, the important and noteworthy distinction resides in the fact that following a heuristic program does not require the animal to evaluate the functional output of the object being manufactured. The cricket does not evaluate the increase in song loudness. The cricket must instead evaluate the structural features, e.g., leaf size, hole size, and position, of the object it is manufacturing. The heuristic process encodes the optimal features of the manufactured object.

These optimization strategies are also not necessarily permanent. Animals, including small brained insects, can use learning, or indeed in some cases rule-abstraction (Avarguès-Weber et al., 2012) to transition from a convergent strategy to one that is heuristic. To explain how such a transition might function, we can use a purely hypothetical example: a hypothetical cricket that like the mole-cricket makes burrows to amplify its sound may make several burrows in its lifetime. It optimizes each burrow by evaluating its sound output. However, this cricket may, through experience, learn optimal burrow dimensions. In such a case, we expect that the starting dimensions of new burrows built by this male cricket would be closer and closer to the optimal burrow size indicating learning. Rule abstraction has indeed been observed in insects (Giurfa et al., 2001; Avarguès-Weber et al., 2012; Avarguès-Weber and Giurfa, 2013) but is much harder to establish, and often requires cleverly designed experiments. To provide an entirely hypothetical example here, we use a hypothetical cricket that makes a baffle similar to that made by tree-crickets. In the rule abstraction case, the cricket may make test baffles in two or more leaves and develop an abstract understanding that the larger leaf is always louder. In subsequent attempts at baffle manufacture, this cricket would chose the larger leaf to make the baffle. The best-test of true rule abstraction in this scenario would be to offer this hypothetical cricket the largest leaf it has previously encountered and one even larger than it. We expect that if this hypothetical cricket has abstracted the rule, it would always chose the larger leaf. However, one that has learned through experience and developed a preference for a particular leaf size, would be more likely to build in the smaller, but previously encountered leaf.

The tree cricket heuristic discussed here has one rule per problem dimension, i.e., a separate rule for every decision that has to be made. There are other, more generalized heuristics, such as the well-known genetic or evolutionary algorithms, the take-the-best algorithm, the diffusion model, and insect-inspired algorithms based on ant colony optimization (Bonabeau et al., 2000; Hutchinson and Gigerenzer, 2005; Cormen et al., 2009; Marshall et al., 2009). These algorithms use a smaller set of rules, and seek to be independent from the problem dimensions. Such heuristics have been developed to solve problems in optimization which cannot be solved analytically in a reasonable length of time. For high dimensional problems, these are the only available methods for optimization that can be accomplished at reasonable speed. While they are certainly faster, we can think of no a priori reason to believe that they will outperform iterative-convergent methods in accuracy. Given the diversity of these heuristics, we suggest that each heuristic must be carefully considered for the optimization problem at hand before such a decision can be made as to what optimization method, or methods, are at work and require testing. This decision is part of the research questions emerging in our own search to understand problemsolving in animals. More problematically, some of these methods need to sample the objective function during the search. We believe heuristics that require a sampling of the objective function should really be considered to be a subset of iterative-convergent processes, but those with specific rules that direct how the objective function should be sampled. Such heuristics essentially enhance the search process as compared to a random walk.

In optimization theory, the conventional wisdom is that iterative-convergent methods are more fine-grained and accurate at reaching optima and that non-convergent methods are essentially compromises that necessarily involve a lack of accuracy (Tversky and Kahneman, 1974). Within the context of optimization theory, finite terminating methods were specifically developed to save time, and heuristics were developed to improve search efficiency whether through saving time or through reducing computational and memory demands. Yet both methods are thought to sacrifice accuracy. Recently, however, we were able to show in a biological object-use system that a heuristic optimization method outlined for baffling-making tree-crickets would always outperform an iterative method in accuracy because the nature of the object being made disallows the animal from 'editing' the object (Mhatre et al., 2017). Here, we will consider other animal object-use systems and delineate (1) the conditions under which heuristic optimization can perform better not just in speed but also in accuracy and (2) those under which the more fine-grained iterative-convergent optimization methods would prevail.

## OBJECT PROPERTIES AND MANUFACTURING TECHNIQUES: EDITING OBJECTS

Iterative-convergent optimization requires that the optimized object be made stage by stage, and that the objective function be measured after each stage of manufacture. If the animal has 'traveled up' the objective function it is expected to continue in that direction, and it must be able to reverse direction if it has 'traveled down.' This is likely to imply, accepting process reversibility, that the animal must be able to undo changes it finds to be detrimental to object function. Such 'editing' may not be possible for all objects.

Some objects are made by a subtractive process, where a part of the object is removed to enable function. These objects are difficult to edit. Re-joining a removed part usually requires new manufacture techniques such as gluing (weaver ants and their larvae), stitching (tailor birds), or lashing with fibers (bagworm moth larvae). Such techniques may be unavailable to the animal, or be feasible only at some stages of manufacture. Tree cricket baffles, which serve the function of enhancing sound production, are made by cutting a hole in a leaf through which the animal sings (**Figure 1**). The objective function value of a particular baffle design is only meaningful with a completed hole. Once

the baffle hole is made, it cannot be erased or 'moved' using the manufacture processes available to the tree crickets. Such editing would require that the crickets glue leaf pieces back, or weave a sheet across the hole, neither of which is an option for them. The crickets can only make a new hole if a current hole is acoustically suboptimal. In addition to the increased cost of manufacture, multiple holes in a single baffle leaf does compromise the acoustic advantage offered by the baffle (Mhatre et al., 2017). Thus, progressive optimization, while possible in baffle manufacture, is likely to be suboptimal compared to a relatively accurate heuristic if, as in this case, process reversibility is not afforded.

Even with additive processes, considerations of structural stability and the ability of animals to manipulate the required materials appropriately will limit how much a completed object can be edited. Adding or removing materials is only possible if the existing structure remains mechanically stable after the change, a problem akin to structural engineering. Additionally, the animal must be capable of making the required changes; for instance, it may be difficult to remove materials that harden after construction. The concept of irreversibility in the manufacturing process plays therefore a role in the domain of opportunities available and thus the evolution of behavioral strategies. Termite nests are designed to balance the dual needs of thermal regulation and gas exchange (Korb, 2003). Their architecture depends strongly on the environmental conditions (Korb, 2007, 2011; Hansell, 2007); savanna mounds have thinner walls and a complex external structure. Forest mounds have thicker walls, and a simple dome-like structure. When the environmental conditions of mounds were experimentally swapped, termites added complex external structures to the domed forest mounds, but did not remove pre-existing structures in the complex savanna mounds (Korb and Linsenmair, 1998).

#### ORGANISMAL PROPERTIES: SENSORY SYSTEMS AND MEMORY LIMITATIONS

An iterative-convergent search method requires that the animal senses some stimulus that accurately reflects the objective function being optimized. Thus, the limitations of a sensory system, or the integrated detection envelope of sensory systems working together, can constrain the accuracy of optimization. For instance, a sensory system with low resolution in intensity coding would impair the detection of changes in the objective function, effectively equivalent to smoothing the objective function. This would result in the animal failing to identify optima altogether. Another example is based on the Weber–Fechner psychophysical law that applies to most sensory systems. The Weber–Fechner law predicts that the smallest perceivable change in a stimulus is proportional to the stimulus magnitude (Kingdom and Prins, 2016). Thus, when the stimulus intensity is already high, only a sufficiently large change in that stimulus is detectable. If and when seeking optimality, a sensory system needs to be able to detect small changes in the objective function, especially as the animal approach optimal values. Thus, an iterative-convergent search method may converge towards the optimal value, yet lose its resolving power whilst doing so, and only attain a sub-optimal value. An elegant example of such sub-optimal performance was recently shown in the case of the bat-bromeliad co-evolutionary dyad (Nachev et al., 2017). Bats were not able to discriminate between higher volumes or even concentrations of nectar, leading to an evolutionary persistence of plants with low quality nectars (Nachev et al., 2017).

In terms of object manufacture, an excellent example are mole crickets making specialized burrows that act as resonators and increase the intensity of their mating calls (Bennet-Clark, 1987). Based on the gradual improvement observed in burrow resonance, mole crickets are thought to be using an iterativeconvergent search method to find optimal burrow dimension and geometry (Bennet-Clark, 1987). Three possibilities are suggested for the possible sensory cues used by the cricket; sound frequency, sound amplitude, and/or perhaps cuticular strain sensors that somehow monitor power output (Daws et al., 1996). Whatever the sensory mechanism used, a closer look at the burrow acoustics suggests that while burrow acoustics improve dramatically in loudness, they do not reach optimal tuning or loudness (Daws et al., 1996). The reason invoked is that sensory systems are not capable of, owing to Weber–Fechner law, reliably coding small changes in large stimuli.

In contrast, using a heuristic method with three simple rules would allow tree crickets to make an acoustically optimal baffle (Mhatre et al., 2017). So, why don't mole crickets use a heuristic method? There may be several reasons for this, an important one is that it may not be possible to abstract the objective function for a burrow's efficiency into a simple set of rules. The objective function of baffles has only three dimensions, is relatively smooth, and the optimization procedure can be coded by a rule per problem dimension as mentioned before (Mhatre et al., 2017). The mole cricket burrow optimization problem has a higher dimensionality since the value of several independent architectural features have to be determined, such as bulb length, bulb diameter, horn length, horn diameter, horn throat/constriction diameter, exit tunnel diameter, and excitation frequency (Daws et al., 1996). A specialized heuristic, such as the one used by tree crickets, that captures the optimal position on specific objective function will have at least one rule per dimension. Notably, if an identified parameter makes no difference to the objective function, the dimensionality of the problem is effectively reduced. On the other hand, if the shape of the objective function is complicated, more rules may be required. For instance, a conditional rule can exist that changes the rule for one parameter dimension depending on the value of another dimension. [For an arbitrary function y = f(x) if parameter x lies between 0 and 1, then choose the largest y, else if x is greater than 1, then choose the smallest y]. For a neural system adapted to handle several sensory modalities, encoding a large number of rules will run into the limit of the animal's memory capacity. For instance, for the mole cricket burrow based on identified, but perhaps not complete, architectural features, we expect that at least seven rules are required. It is also expected that an iterative-convergent method will be computationally lighter and more efficient as the number of rules grow. Thus, iterative processes which are conventionally considered 'higher cognition'

may actually be a strategy for reducing the rules that must be remembered and followed, thus minimizing computational and memory demands (McFarland, 1991).

Do sensory systems play no role in heuristic optimization? Indeed, they do; the heuristic decision making process is usually supported by some sensory information. For instance, in the tree cricket case, a larger leaf is chosen, a decision that requires information about leaf size. Additionally, tree crickets also size the hole with respect to their own wing size, and center it within the leaf. This process requires sensory information which enables them to size their wings and the hole and to find the leaf center. The important distinction between this heuristic and an iterative process, however, is that the cricket does not sense the functional output of the baffle (loudness), but rather its architectural features. Using the sensory system, however, means that errors can arise even in heuristic optimization. In the tree cricket system, errors have been observed in centring the baffle hole (Mhatre et al., 2017). In general, for the heuristic to outperform or equal iterative optimization, the performance-cost due to errors in these heuristic decisions must be lower than errors accrued from estimating the objective function stimulus directly.

#### ORGANISMAL PROPERTIES: COLLECTIVE BEHAVIOR

Insects are fairly unique among animals; through collective behavior, they can make objects that are made out of their own bodies. Remarkably, ants, bees, and wasps are deemed to make as many as 18 different kinds of self-assemblages (Anderson et al., 2002). A few examples encompass bridges that help them traverse gaps (Reid et al., 2015), rafts that enable them to survive in flood plains (Mlot et al., 2011), force generating clamps to hold the edges of a leaf together to sew them into a brood tent (Holldobler and Wilson, 1983), and anti-predatory 'ovens' which bees make by 'balling' around hornets to kill them by overheating (Ono et al., 1995).

Despite their difference from other objects we have considered so far, self-assemblages are also likely to be optimized. Since the assemblages do not use external objects or materials, selection remains confined to the insect's cognition and morphology. Additionally, the assemblages serve crucial functions and are unlikely to be 'spandrels.' There is evidence for optimization in features such as a balancing of cost-benefit ratios in ant bridges (Reid et al., 2015; Graham et al., 2017), and very small tolerances for the temperatures achieved by the bee ovens which would kill the bees themselves if it increased by ∼2–4◦C (Ono et al., 1995).

For collective structures, it is known that individuals change behavior and make decisions at speeds that preclude their being directed by a co-ordinating or leading individual (Couzin, 2007, 2009). Thus, there is no individual or entity that coordinates and monitors the performance of the object. Rather, the overall structure emerges from the decisions made by each individual based on simple rules which respond only to local information, i.e., via a heuristic (Anderson, 2002; Couzin, 2009).

Insects do make other types of objects through collective behavior, most notably the impressive habitations of the social insects – termite mounds, ant and wasp nests, and bees hives. These structures typically tend to be multifunctional and must fulfill the multiple demands made on habitations: providing protection from the external elements, against pathogens, parasites, and predators, also ensuring good climate control and ventilation, and fluid transport of goods and individuals within the nest. Thus, the objective function of these structures is likely to be more complicated and to summate these properties in a weighted fashion. Given the complexity of these structures, their size and the fact that they often grow continuously, it is also difficult to find the parameters that adequately describe the input space of these objects. Despite these difficulties, there has been some remarkable work recently in studies of collective building (Karsai and Wenzel, 2000; Perna et al., 2008a,b; King et al., 2015; Khuong et al., 2016) and the optimization of collectively built structures has been closely considered (Perna et al., 2012).

Given that there is no 'co-ordinating' individual, how is construction regulated in collectively built structures? As we understand it today, the main mechanisms that guide building are (1) stigmergy, i.e., insects interacting with the structure and (2) direct interactions between the insects themselves (Downing and Jeanne, 1988; Jeanne, 1996; Theraulaz and Bonabeau, 1999; Anderson, 2002). In stigmergic building, the construction behavior of the insect is directed by the structure or some of its features, i.e., the physical object it encounters or some chemical cue within this object can then drive its behavior. For instance, ants building a nest are more likely to deposit a pellet of building material in response to previously deposited pellet which has a high concentration of a pheromone, than next to one with a lower concentration (Khuong et al., 2016), whereas wasps may determine where to build the next cell based on the number of hexagonal sides free on the edge of the current structure (Karsai and Penzes, 1993). Direct interactions may regulate building, in particular in terms of nest size. It is suggested that population density determines the size and to an extent the structure of some ant nests (Franks et al., 1992; Buhl et al., 2004).

Is it possible for insects building structures collectively, whether with their own bodies or using other materials, to also use a different iterative-convergent method for optimizing this structure? The generation of an object by a heuristic approach does not necessarily preclude iterative-convergent optimization. In effect, as long as the object can be changed progressively, the optimization process can be separate from the construction process. However, the lack of a 'co-ordinating' individual does seem to prevent iterative-convergent optimization since there would be no examination of the global objective function and subsequent directing of behavior. Another possibility, however, is that each individual may sample the objective function, or a section of it, and modulate its building behavior accordingly (Perna et al., 2012). The difficulty in this scenario is that the structure being optimized is usually significantly larger than the insects building it and this limits their perceptual ability (Perna et al., 2012). However, if there were a stimulus that reflected the

objective function, which was relatively homogenous within the structure, this might be a viable possibility. A simple example would be that the builders could monitor the temperature, CO<sup>2</sup> or air-flow inside the nest, if these were relatively gradient free. However, typical nests are structured to generate gradients in these very features and these gradients are exploited to generate air-flows which ventilate, and redistribute heat within the nest (Korb, 2003; King et al., 2015). Another possibility is that the builders use sampling methods to estimate these quantities, as they have been shown to use to estimate nest size (Mallon and Franks, 2000; Mugford, 2001). However, the gradients within the structure are systematic (King et al., 2015), and even the structure of the nest itself is topologically systematic (Perna et al., 2008b). Therefore, simple random-walk based sampling methods would be insufficient and sampling within such structures would likely require an internal 'map' of the nest and spatial awareness. Cues such as nest temperature, humidity, airflow direction do modify insect building behavior (Korb and Linsenmair, 1998; Bollazzi and Roces, 2007, 2010b), but they are more likely to guide the modification of structures rather than the initial construction. We tackle this issue more completely in a later section.

In self-assemblages, the issue of information acquisition seems somewhat clearer. It is likely that the insect which is participating in the structure is likely to have access to only very local information and cannot access the global efficiency of the structure (Anderson, 2002; Anderson et al., 2002). For instance, in ant rafts, individual ants assemble to make a structure that floats because it is both buoyant and water repellent. (Mlot et al., 2011). While the rafts are made well enough to prevent even the ants on the bottom from drowning, the ants on the edges, bottom and in the center of the raft support different weights and have different oxygen supplies available to them (Mlot et al., 2011; Foster et al., 2014; Tennenbaum et al., 2016). Even where the behavior is purely mechanical, relying on some simple homogenous bulk quality such as the stiffness of the aggregate (Tennenbaum et al., 2016), such as in bridges or ropes, the forces experienced by animals at the edges and boundaries, will be different from those in the center, suggesting that local cues will differ from global cues preventing iterativeconvergent optimization aimed at individual ants (Anderson, 2002). In general, we surmise that collective structures built using heuristic techniques are probably optimized in a similar fashion.

# PROBLEM PROPERTIES: CHANGING OBJECTIVE FUNCTIONS

All the cases we have considered so far have static objective functions, i.e., the efficiency of an object of a particular design remains constant. In the real world, however, the efficiency of a particular object design may change as the object interacts with a changing environment, such as the changes in a nest's efficiency as the light-shade regime changes or with changes in temperature, or in rainfall and humidity. Objective functions could also change when the object interacts with other organisms, for instance, traps. This could happen either through a slow evolutionary process such as an arms race between the trap maker and prey, or through faster processes such as the species composition and number of organisms the trap interacts with changes seasonally. Strictly speaking, environmental or organismal variables are not characteristics of the object, and hence cannot be incorporated as a dimension of the object's state space. From the biological standpoint, however, optimization would require the animal to adapt the object and change its design to suit the changed conditions.

Invertebrate prey-capture traps come in two broad categories of design, pitfall traps or a web-based trap (Hansell, 2007; Scharf et al., 2011). Trap building involves structural considerations, for instance spider-webs need to be robust to environmental damage (Cranford et al., 2012; Sensenig et al., 2012; Qin et al., 2015), and functionally, these constructed objects may be used for other purposes such as mating rituals (Vibert et al., 2016). Their primary purpose, however, is food acquisition. Spider webs are complex structures with a wide variety of designs ranging from the commonly encountered two dimensional orb webs to the rarer three dimensional cobwebs of black widow spiders (Blackledge et al., 2009). It would be challenging to create a single and complete analytical framework to examine the entire range of web designs. Nonetheless, several authors have identified three functional features that are crucial to understanding trap efficiency: the ability to intercept, stop, and retain prey (Eberhard, 1990; Blackledge et al., 2011). To intercept prey, the traps must efficiently cover their capture area with silk, and make a web of appropriate mesh size. This web should either be relatively inconspicuous to prey or actually be attractive to prey. Next, to stop prey, webs must efficiently dissipate the kinetic energy imparted at prey impact without breaking or bouncing the prey off the web. This problem is largely addressed through different types of silk extruded by the spiders. Finally, the web must retain the prey, either by adhesion or by entanglement, another problem usually solved by using distinct silk types and, occasionally, structures such as ladders. It is known that different web designs have different efficiencies for each of these processes, and that spiders change their trap structure in response to their prey capture rate and nutritional status. Thus far, however, given the complexity of the problem much of the work addresses only a few features of spider web efficiency at a time (Eberhard, 1990; Blackledge and Zevenbergen, 2007; Zevenbergen et al., 2008; Blackledge et al., 2011; Blamires et al., 2016). Pitfall traps are simpler than webs. Among the most familiar are the pits of antlion larvae, which are conical depressions in loose sandy soil with the antlion hidden near the cone's apex. The main features of the pit are its location, width, the slope of the walls and the particle size of the soil, and these features together determine the size of the prey captured and the likelihood that it will slip down the pit slope (Devetak et al., 2005; Fertin and Casas, 2006). Remarkably, the slope of the trap is optimized so as to be on the verge of the critical point of stability of the particular sand granularity, where a slight disturbance is poised to generate an avalanche leading the prey to the ambushing predator (Fertin and Casas, 2006). With pitfall traps, at least in some cases, trap size and structure appear to be also optimized

for certain prey species (Devetak et al., 2005; Barkae et al., 2012).

How might traps be optimized given that the objective function of a particular design might change over its lifetime? The main theory covering optimization in this context is optimal foraging theory and there is some evidence that insects have the behavioral flexibility to optimize their foraging strategy (Scharf et al., 2011). If the approach to optimization remains purely heuristic, then the animal must switch between different rule sets for distinct functions. In addition, the animal has to sense some hallmark stimulus that indicates the transition from one objective function to another and chose the appropriate objective function for the transition. This leads to a problem that has been noted before: a large number of rules would have to be encoded into the heuristic (McFarland, 1991). In this situation an iterative convergent strategy might perform better. Indeed it has been reported that trap builders either evaluate trap efficiency directly, or through their own nutritional status, and use this imperfect information to guide trap modifications such as changing its size, shape, components, or location (Scharf et al., 2011). Such a process is suggestive of an optimization strategy that is iterative, rather than based on a bank of heuristic routines. However, both web or pitfall traps tend to have low and highly variable capture rates, whereby some traps catch nothing over several days (Edwards et al., 2009). This unpredictability is at the heart of the question of optimization, and makes it difficult to accurately assess trap efficiency (Blackledge et al., 2011; Scharf et al., 2011). Thus, even an iterative strategy may not be able to approach optimal design, and achieve the theoretically optimal foraging strategy. Interestingly, exit strategies exist; as traps are likely to be abandoned following trap damage, parasitic invasion, or competition, none of which relate to trap efficiency itself (Blackledge et al., 2011). Thus, trap or web abandonment may be indicative of boundary conditions of the objective function, or neighboring objective functions, and help test decision mechanisms and the logic of state-dependant transitions between strategies.

Unlike variations in prey distributions in time and space, environmental variations may be considered more predictable since they are often brought about by circadian or seasonal rhythms. At the local scale of the ecological niche of small animals, uncertainty in both trophic and abiotic factors prevails, constituting part of the challenge in the search for optimality. Nest building insects seem to have developed both active and passive mechanisms for dealing with variation and uncertainty (Jones and Oldroyd, 2006). The most common passive adaptation for dealing with temperature variations is nest insulation which helps maintain a steady internal nest temperature. This tolerance is achieved by several mechanisms, such as multi-layered insulation within the nest structure as observed in stingless bee species (Roubik, 2006), or by orienting the nest with respect to the sun in order to harness solar heating as observed in magnetic termite mounds (Jacklyn, 2010). As these variations are of the type that can be anticipated, the nest can be structured, and actively modified, with available manufacture methods. For instance, some ants regulate nest temperature and humidity by plugging and unplugging air vents which are made only to regulate temperature and humidity and do not function as entrances (Franks, 1989; Bollazzi and Roces, 2007, 2010a,b). Another common mechanism for dealing with environmental changes, such as seasonal changes is simply to make shortlived nests and abandon them for new nests developed for the newer conditions, a behavior seen in ants that build different winter and summer nests (Ofer, 1970). Bees and wasps are well known to heat up their nest by using metabolic heat generated by rapidly contracting their flight muscles (Kronenberg and Heller, 1982). Inversely, bees can cool their nest down, whereby water deposited on the nest surface evaporates and serves to lower nest temperature (Ishay and Barenholz-Paniry, 1995). Arguably, the most intriguing passive thermoregulatory mechanism uses thermoelectric material properties; the silk of pupal cases in the hornet appears to accumulate electric charge under hot conditions, and releases it during cold conditions and helps maintain pupal temperature (Ishay and Barenholz-Paniry, 1995).

Given the wide range of mechanisms, it is difficult at the moment to make a single argument for whether the adjustment of optimization points is carried out heuristically, or using an iterative-convergent method, or a mix thereof. A few different possibilities exist: where passive insulation is important, no new behavior is required; where new structures are built, they may be built using a different set of optimized rules with the cue for the shift being a seasonal rather than nest-based cue, such as day–night length. This would connote a heuristic process of re-optimization. Where the nest is modified in some fashion, however, the insects must receive some nest-efficiency cue that initiates the re-optimization procedure. Whilst such a cue may be sensed by individual insects, and may result in a new emergent collective behavior that seeks a novel optimum, it may perhaps never reach this optimum, but might nonetheless adapt to a novel state space. Such a system dynamics view may be useful to experimentally identify key cues that prompt such transitions. Quite certainly non-linear, such transitions may be the key to the presence of adaptive fast heuristics. However, as we have discussed in the context of collective construction, cues within the nest are variable, and more importantly vary systematically. As individual insects are likely not to have access to a global measure of efficiency of the nest, it may be useful to hypothesize that local conditions ought to provide sufficient information that locally engages many individuals into the proper heuristics. Of course, these considerations are not limited to insects but also encompass vertebrates that build collectively, such as the African social weavers (Van Dijk et al., 2013). One of the ways that nesting insects solve this problem is to monitor temperature where it is most crucial. Nest temperature has a large effect on brood development. Thus, local monitoring of the brood chamber and, when necessary, moving brood to other parts of the nest where temperatures are more favorable is an effective strategy (Jones and Oldroyd, 2006). Another possibility is that the variation in nest temperature may not be a significant factor in nest optimization. Indeed, some modeling studies suggest that variance in the sensitivity to temperature in nest building insects may actually be important in stabilizing the temperature within

the nest (Myerscough and Oldroyd, 2004; Graham et al., 2006; Jones and Oldroyd, 2006).

#### CONCLUSION: INTEGRATING MODELS AND BEHAVIOR

The question of how seemingly "complex" behaviors such as object optimization are organized in the so-called "simple" organisms can benefit from a careful disssection of the physical dimensions of the problem and the different approaches available for solving the problem. We suggest here that a basic approach, such as heuristics, can help explain behavioral adaptations without the requirement of large neuronal processing power. This proposition is naturally complementary, and not exclusive, to other solutions employing such brain power. The particulars of object use and optimization is only one of many realms of application, much of it remains to be explored in terms of heuristic optimization. In effect, our analysis of object use in insects has, by itself, implications for how we can gain a more complete understanding of the living world. In particular, there is much to gain by examining the organizational patterns that connect organisms with the physical aspects of their ecology. With this respect, predictive models anchored in the physics of the manufactured objects are needed that can identify objective functions and their key parameters, capture boundary conditions and characterize feasible domains. Such models can directly help

# REFERENCES


formulate testable hypotheses and test behavioral decisions and their consequences (e.g., Perna et al., 2008a,b). In particular, the power of analytical methods traditionally used in engineering, such as finite element modeling and analysis, are increasingly applicable to heterogenous and dynamic biological structures. Used in conjunction with high-resolution X-ray tomography and 3D printing, much insight could be gained from modeling and experimental approaches.

In its own and perhaps small way, this heuristic approach challenges the overused, poorly supported and dysfunctional metaphysical category "simple." Celebrating the power of observation, when considered for long enough, insight ensues and nothing becomes simple.

#### AUTHOR CONTRIBUTIONS

NM performed the original research and data analysis, and led the MS design and writing. NM and DR conceived of the research and wrote the manuscript.

# FUNDING

The own research reported in this article was supported by a grant from BBSRC. Funding for NM was from Wissenschaftskolleg zu Berlin, and for DR from BBSRC (Grant No. BB/M011143), the Royal Society, and the ERC Advanced Investigator Grant.




**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

The reviewer AA-W and handling Editor declared their shared affiliation.

Copyright © 2018 Mhatre and Robert. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Signaling Pathways for Long-Term Memory Formation in the Cricket

#### Yukihisa Matsumoto<sup>1</sup> \*, Chihiro S. Matsumoto<sup>1</sup> and Makoto Mizunami<sup>2</sup>

<sup>1</sup> College of Liberal Arts and Sciences, Tokyo Medical and Dental University, Ichikawa, Japan, <sup>2</sup> Graduate School of Life Sciences, Hokkaido University, Sapporo, Japan

Unraveling the molecular mechanisms underlying memory formation in insects and a comparison with those of mammals will contribute to a further understanding of the evolution of higher-brain functions. As it is for mammals, insect memory can be divided into at least two distinct phases: protein-independent short-term memory and proteindependent long-term memory (LTM). We have been investigating the signaling pathway of LTM formation by behavioral-pharmacological experiments using the cricket Gryllus bimaculatus, whose olfactory learning and memory abilities are among the highest in insect species. Our studies revealed that the NO-cGMP signaling pathway, CaMKII and PKA play crucial roles in LTM formation in crickets. These LTM formation signaling pathways in crickets share a number of attributes with those of mammals, and thus we conclude that insects, with relatively simple brain structures and neural circuitry, will also be beneficial in exploratory experiments to predict the molecular mechanisms underlying memory formation in mammals.

#### Edited by:

Martin Giurfa, UMR 5169, Centre de Recherches sur la Cognition Animale (CRCA), France; Université Toulouse III Paul Sabatier, France

#### Reviewed by:

Maria Eugenia Villar, Fyssen Foundation, France; Université Toulouse III Paul Sabatier, France Lesley J. Rogers, University of New England, Australia

> \*Correspondence: Yukihisa Matsumoto yukihisa.las@tmd.ac.jp

#### Specialty section:

This article was submitted to Comparative Psychology, a section of the journal Frontiers in Psychology

Received: 01 March 2018 Accepted: 31 May 2018 Published: 22 June 2018

#### Citation:

Matsumoto Y, Matsumoto CS and Mizunami M (2018) Signaling Pathways for Long-Term Memory Formation in the Cricket. Front. Psychol. 9:1014. doi: 10.3389/fpsyg.2018.01014 Keywords: long-term memory, NO-cGMP signaling, cAMP signaling, crickets, classical conditioning

# INTRODUCTION

Brain structures and neural circuitry of insects are relatively simple, and they are therefore useful for exploratory experiments to predict the molecular mechanisms underlying memory formation in mammals. Memory in insects as well as that in vertebrates is a dynamic process organized in two main types: short-term memory (STM) and long-term memory (LTM). The former is defined as protein synthesis-independent memory, and the latter is defined as protein synthesis-dependent memory. They can be distinguished by their temporal courses and molecular mechanisms (Kandel, 2001). It is a common understanding that while STM is based on temporal changes in the synaptic strength due to covalent modifications of pre-existing proteins, LTM is supported by long-lasting alteration in the strength of synaptic function demanding for transcription and translation of genes, among a wide variety of animals including mice, sea hares Aplysia and fruit flies Drosophila (Montarolo et al., 1986; DeZazzo and Tully, 1995). The cAMP pathway is demonstrated to be critical for LTM formation in all of these animals (Bartsch et al., 1995; Yin et al., 1995; Abel et al., 1997). The cAMP pathway is a signaling cascade beginning with an increase in intracellular cAMP that activates cAMP dependent protein kinase (PKA). PKA phosphorylates the transcription factor cAMP-responsive element-binding protein (CREB) that leads to LTM formation. The nitric oxide (NO)-cGMP pathway is another system playing critical roles in the formation of LTM in sheep (Kendrick et al., 1997), great pond snails Lymnaea (Kemenes et al., 2002), and honey bees (Müller, 1996, 2000).

In this review, we will summarize the results of our pharmacological behavioral studies on the molecular mechanisms of the formation of LTM in the cricket Gryllus bimaculatus and propose an updated model of LTM formation. The main results introduced in this review are shown in **Table 1**.

Crickets provide several advantages to investigate memoryrelated molecules. First, they demonstrate remarkable ability of olfactory learning and memory, including that requires cognitive functions. For example, they exhibit robust olfactory memory maintained throughout their lifetime (Matsumoto and Mizunami, 2002a), contextual learning (Matsumoto and Mizunami, 2004), high capacity of memory storage (Matsumoto and Mizunami, 2006), second-order conditioning (Mizunami et al., 2009), and sensory preconditioning (Matsumoto et al., 2013a). In addition, they have remarkable visual learning ability (Unoki et al., 2006; Nakatani et al., 2009; Matsumoto et al., 2013b). Second, effective approaches that greatly facilitate analysis of the molecular basis of learning and memory are feasible. Recent progress in genetics allowed establishment of gene knockdown by RNA interference (RNAi) (Takahashi et al., 2009; Awata et al., 2016) and genome editing by the CRISPR/cas9 system (Awata et al., 2015) in crickets, adding to the well-established pharmacological methods (Unoki et al., 2005, 2006; Matsumoto et al., 2006, 2009, 2016; Mizunami et al., 2014; Sugimachi et al., 2016). Third, there has been a good accumulation of knowledge that bridges between the nervous system and behavior of crickets gained by extensive neuroethological studies in crickets (Stevenson and Schildberger, 2013; Hedwig, 2016).

#### Experimental Procedures

In our previous works in crickets, we have developed and extensively studied the olfactory associative conditioning, in which an odor is paired with reinforcement stimulus (Matsumoto and Mizunami, 2000, 2002b; Matsumoto et al., 2015). Similar conditioning protocols applied to two different types of visual stimuli, visual-pattern (Unoki et al., 2006) or color-vision (Nakatani et al., 2009), paired with reinforcement stimuli have also been established. All of these procedures use classical conditioning for training and operant testing for memory tests (Matsumoto and Mizunami, 2002b; Matsumoto et al., 2003) and is performed on individual, isolated cricket. This protocol is built on the fact that crickets are able to transfer memory formed by classical conditioning in a beaker, half-compelled to receive the training, to the environment that allows freedom of choice in a larger testing chamber.

We will slightly go through the details of conditioning taking olfactory appetitive conditioning of an odor with water reward as an example. Before the experiment, crickets are each isolated in a beaker without water for 3 days, which enhances water consumption. A syringe containing water with a piece of filter paper set near the needle tip is used in conditioning training. Odor essence is applied to the filter paper to present the odor. The cricket receives the odor around its antennae for 3 s, and then receives a drop of water reward to the mouth. On water application, crickets attempt to drink it indicating that water serves as an appetitive stimulus. Retention scores of memory formed by single pairing of an odor with water reward (single-trial conditioning) is as high as that formed by repeated pairings of odor-reward association (multiple-trial conditioning) at 30 min after training, but it declines over a period of several hours and is no longer observed at 1 day after training (Matsumoto et al., 2006).

Multiple-trial conditioning consist of two or more repetition of odor-reinforcement trials with inter-trial intervals (ITIs) that induces long-lasting memory beyond 1 day under adequate


NOS, NO synthase; sGC, soluble guanylyl cyclase; CNG channel, cyclic nucleotide-gated channel; CaM, calmodulin; AC, adenylyl cyclase; PKA, protein kinase A; L-DIL, <sup>L</sup>-cis-diltiazem; DDA, 205 0 -dideoxyadenosine; CHX, cycloheximide. Data for L-NAME, L-DIL, and W-7 experiments are from Matsumoto et al. (2006); Data for ODQ, DDA and KT5720 experiments are from Matsumoto et al. (2006, 2009); Data for KN-62 experiments are from Mizunami et al. (2014); Data for CHX experiments are from Matsumoto et al. (2003, 2006). The concentrations of the administrated drugs were as follows: L-NAME (400 µM), ODQ (200 µM), L-DIL (1 mM), W-7 (200 µM), KN-62 (2 mM), DDA (1 mM), KT 5720 (200 µM), CHX (10 mM), NO-donor SNAP (200 µM), cGMP analog 8-br-cGMP (200 µM), Ca2<sup>+</sup> ionophore A23178 (200 µM), AC activator forskolin (200 µM), cAMP analog 8-br-cAMP (200 µM), and DB-cAMP (200 µM).

conditions (e.g., number of trials = 4, ITI = 5 min). Multipletrial conditioning in our previous studies includes absolute conditioning (A+) and differential conditioning (A+, B−). Absolute conditioning can be described as repetition of appetitive conditioning trials. Differential conditioning combines appetitive and aversive conditioning trials in an alternating order. For olfactory aversive conditioning of an odor with sodium chloride punishment, similar syringe containing 20% sodium chloride solution is used. The crickets show immediate retraction from sodium chloride solution, indicating that it functions as an aversive stimulus. In previous works, we used differential conditioning that leads to robust memory (Matsumoto et al., 2006), but we eventually switched to absolute conditioning for the simplicity of analysis (Matsumoto et al., 2009; Mizunami et al., 2014).

Before and after olfactory associative conditioning, crickets were tested for their odor preferences between two odors during a 4 min testing period. Tests were performed operantly, allowing a cricket to search and choose from two odor sources, a control odor and a conditioned odor, provided in the testing chamber. Relative odor preference index for each cricket was calculated from the visiting time for each of the odor sources, as a ratio of rewarded-odor visiting time to the total visiting time. Visiting time was recorded when odor source was explored by the mouth parts of the cricket.

In our pharmacological behavioral experiments, basically, we injected 3 µl of saline containing a drug into the hemolymph of the cricket's head using a microsyringe 20 min before the onset of training (see **Table 1** legend for drug doses). All of the drugs used in our experiments had been confirmed for their efficacy in physiological or biochemical researches in insects.

#### Memory Phases

As is the case with other animals (DeZazzo and Tully, 1995), memory induced by multiple-trial conditioning in crickets can be further distinguished into several memory phases with different retention curves. In our previous work applying differential conditioning in crickets, we have demonstrated that olfactory memory can be subdivided into at least two memory phases, STM and LTM. The peak memory score induced by sufficient multipletrial conditioning with sufficient ITIs is retained without decline for a few days (Matsumoto and Mizunami, 2002b), but when injected with a protein synthesis inhibitor (e.g., cycloheximide), memory retention score started to diminish from 5 h after training, and completely disappeared at 8 h after training (Matsumoto et al., 2003). The results indicate that there are two types of memory phases discriminated by the sensitivity to a protein synthesis inhibitor. One type is named LTM that requires protein synthesis and at least maintained for several days (Matsumoto and Mizunami, 2002b). The other type is STM which does not require novel protein synthesis (Matsumoto et al., 2003). The STM peaks immediately after the training until 4 h after training and disappears at 8 h after training. Differential conditioning may be a rather complicated learning task involving both appetitive and aversive learning. Thus, we are switching the conditioning paradigm to the simpler absolute conditioning in recent works. The memory phases in absolute conditioning should be clarified by further investigation.

#### cAMP Signaling Pathway

The cAMP signaling system has been demonstrated to be essential in LTM formation in mice (Abel et al., 1997), Drosophila (Yin et al., 1995; Isabel et al., 2004) and Aplysia (Bartsch et al., 1995). LTM formation in all of these species requires phosphorylation of transcription factor CREB (cAMP-responsive element-binding protein) by PKA (cAMP-dependent protein kinase) which is activated by an increase of intracellular cAMP (Bartsch et al., 1995; Yin et al., 1995; Abel et al., 1997).

We investigated whether cAMP signaling is necessary for LTM formation in the cricket (Matsumoto et al., 2006, 2009). Crickets were each injected with inhibitors of key enzymes of cAMP signaling into the hemolymph prior to multiple-trial conditioning. We used either 2<sup>0</sup> ,50 -dideoxyadenosine (DDA) or SQ22536 as an adenylyl cyclase (AC) inhibitor, and either KT5720 or Rp-8-br-cAMPS as a PKA inhibitor. In a retention test 1 day after training, all of the groups of crickets failed to exhibit increased preference to the conditioned odor in comparison to that before conditioning (**Figure 1**). On the other hand, they showed normal scores of 30-min memory retention similar to the control group that had received injection of cricket saline. These observations indicate that these drugs fully impair LTM formation but have no effect on STM formation, motivation, sensory or motor functions. On the other hand, when these drugs were administered after conditioning, they did not impair LTM, indicating that it is during conditioning that activation of cAMP signaling is necessary for LTM formation.

The results of our experiments using 'LTM-inhibiting' drugs showed that cAMP signaling is necessary for LTM formation

in the cricket, but is it also sufficient for LTM formation? To address this issue, we investigated whether forced LTM formation occurs by upregulating the cAMP signaling pathway during single-trial conditioning, which does not form LTM. Crickets were each injected with an AC activator (forskolin) or a cAMP analog (DB-cAMP, 8-br-cAMP) into the hemolymph prior to single-trial conditioning. In a retention test 1 day after the conditioning, higher preference scores for the conditioned odor in comparison to that before conditioning were observed in all of the groups, and their scores were as high as that of crickets that had been trained by multiple-trial conditioning (Matsumoto et al., 2006; Mizunami et al., 2014). Moreover, crickets co-injected with a protein synthesis inhibitor (cyclohexymide) and one of the activators of cAMP signaling paired with single-trial conditioning did not exhibit 1-day memory retention. These results suggest that activators of cAMP signaling induce protein-dependent LTM.

#### NO-cGMP Signaling Pathway

NO-cGMP signaling is also critical for producing LTM in sheep (Kendrick et al., 1997), Lymnaea (Kemenes et al., 2002) and honey bees (Müller, 1996, 2000). NO is both intraand intercellular signaling molecule with high reactivity and membrane-permeable property, synthesized by NO synthase (NOS). Through paracrine effect of NO, soluble guanylyl cyclase (sGC) in adjacent cells produce cGMP which is involved in various physiological functions (Garthwaite et al., 1988; Garthwaite and Boulton, 1995), including induction of LTM in many animals (Bernabeu et al., 1996; Prickaerts et al., 2002).

To investigate whether NO-cGMP signaling is necessary for LTM formation in the cricket, crickets were each injected with an NOS inhibitor (L-NAME) or an sGC inhibitor (ODQ) prior to multiple-trial conditioning (Matsumoto et al., 2006, 2009). These groups of crickets did not show 1-day memory retention, whereas 30-min memory retention remained intact (Matsumoto et al., 2006, 2009). These observations indicate that inhibition of NO-cGMP signaling fully impairs LTM formation but has no effect on STM formation. We also obtained comparable results using RNAi: injection of NOS dsRNA fully impaired 1-day retention but not 30-min retention in 7th-instar nymphal crickets (Takahashi et al., 2009).

The results of our experiments using 'LTM-inhibiting' drugs showed that NO-cGMP signaling is required to establish LTM in the cricket. Next, we investigated whether externally applied activators of NO-cGMP signaling paired with singletrial conditioning can facilitate LTM formation. Crickets each injected with an NO donor (SNAP, NOR3) or a cGMP analog (8-br-cGMP) before the single-trial conditioning showed significantly high retention level at 1 day after conditioning, which was almost identical to that in saline-injected group at 1 day after multiple-trial conditioning (Matsumoto et al., 2006). Moreover, crickets co-injected with a protein synthesis inhibitor (cyclohexymide) and an activator of NO-cGMP signaling paired with single-trial conditioning did not exhibit 1-day memory retention, indicating that activators of NO-cGMP signaling pathway induce formation of protein-dependent memory, that is, LTM.

# NO-cGMP Signaling Stimulates cAMP Signaling to Induce LTM

Our pharmacological behavioral experiments using 'LTMinhibiting' drugs or 'LTM-inducing' drugs suggested that NOcGMP signaling and cAMP signaling are both necessary and sufficient for cricket LTM formation, particularly in the conditioning process.

Next, to determine which of the two pathways, NO-cGMP signaling or cAMP signaling, precedes the other in the LTM formation cascade, we varied the combinations of 'LTMinhibiting' drugs or 'LTM-inducing' drugs paired with singletrial conditioning and evaluated their effects. For example, we investigated whether cAMP mediates the forced LTM formation by combining a cGMP analog injection with singletrial conditioning (Matsumoto et al., 2006). While LTM induction by combination of a cGMP analog (8-br-cGMP) and single-trial conditioning was unaffected by co-injection of an NOS inhibitor (L-NAME), it was completely impaired by co-injection of an AC inhibitor (DDA).

Induction of LTM by single-trial conditioning paired with 'LTM-inducing' drugs related to cAMP signaling (AC activator forskolin, cAMP analog DB-cAMP) was unaffected by 'LTMinhibiting' drugs related to NO-cGMP signaling (L-NAME, ODQ) (Matsumoto et al., 2006). In contrast, induction of LTM by single-trial conditioning paired with 'LTM-inducing' drugs related to NO-cGMP signaling (SNAP, 8-br-cGMP) was fully blocked by 'LTM-inhibiting' drugs related to cAMP signaling (DDA, KT5720). The results suggest that in the LTM induction process, the AC-cAMP pathway works downstream of the NOcGMP pathway, and not vice versa.

## Biological Pathways Intervening Between NO-cGMP Signaling and cAMP Signaling

Next, we investigated biological pathways intervening between cGMP and AC activation. PKG, a cGMP-dependent protein kinase, is one of the possible targets of cGMP. Working in parallel with PKA, PKG enhances the phosphorylation of CREB in mice (Lu and Hawkins, 2002). Working in parallel with the cAMP pathway, NO-cGMP-PKG signaling pathway governs the induction of long-term hyper-excitability on receiving a noxious stimulation in nociceptive sensory neurons of Aplysia (Lewin and Walters, 1999). We investigated the roles of PKG in olfactory memory in the cricket. LTM formation was not affected by external application of PKG inhibitor KT5823, whether it was induced by multiple-trial conditioning or by single-trial conditioning combined with 8-br-cGMP.

Thus, we switched our target to cyclic nucleotide-gated cation channel (CNG channel). CNG channels are Ca2+-permeable channels activated by cAMP and/or cGMP. A CNG channel inhibitor [L-cis diltiazem (L-DIL), 3,4,-dechlorobenzamil (DCB)] fully impaired LTM, but not STM, formed by multiple-trial conditioning. Moreover, the CNG channel inhibitor L-DIL fully impaired LTM induced by combination of a cGMP analog (8-br-cGMP) and single-trial conditioning, while L-DIL did not affect LTM induced by 'LTM-inducing' drugs related to

cAMP signaling (forskolin, DB-cAMP) paired with single-trial conditioning. From the results, it can be suggested that CNG channel plays its role downstream of cGMP and upstream of AC activation in the LTM formation process.

In Drosophila, it has been shown that AC is activated by either G-protein or calcium-calmodulin (Ca2+/CaM) (Livingstone et al., 1984). CaM is a principal Ca2+-binding messenger protein in the central nervous system. We examined whether CaM mediates the signaling pathway from CNG channel to AC activation. A CaM inhibitor (W-7) fully impaired LTM formed by multiple-trial conditioning. Moreover, the CaM inhibitor W-7 fully impaired LTM induced by a cGMP analog (8-brcGMP) paired with single-trial conditioning, while it had no effect on LTM induced by 'LTM-inducing' drugs related to cAMP signaling (forskolin, DB-cAMP) paired with single-trial conditioning. Next, we investigated whether rise in calcium concentration mediates signaling from CNG channel to CaM in LTM formation process. Crickets injected with a calcium (Ca2+) ionophore (A23187) paired with single-trial conditioning exhibited LTM. The LTM induced by A23187 was unaffected by co-injection of an sGC inhibitor (ODQ) or a CNG channel inhibitor (L-DIL) but was completely impaired by co-injection of a CaM inhibitor (W-7) or an AC inhibitor (DDA). The results indicate that Ca2+/CaM mediates signaling from CNG channel to AC, filling the gap of LTM formation cascade.

Ca2+/CaM-dependent serine/threonine kinase II (CaMKII), which is one of the Ca2+/CaM effector enzymes, supports various learning and memory systems as a key signaling molecule in vertebrates (Coultrap and Bayer, 2012). This is especially because CaMKII have the ability to modulate its own kinase activity by autophosphorylation. In the fruit fly Drosophila, synthesis of CaMKII in mushroom bodies has been reported to be necessary for olfactory LTM formation (Ashraf et al., 2006; Akalal et al., 2010; Malik et al., 2013). The mushroom body is known as a multisensory association center as well as a secondary olfactory center essential for olfactory learning and memory (Heisenberg, 2003; Davis, 2011). In cockroaches, an increase of phosphorylated CaMKII is observed in pre- and post-synaptic structures in the mushroom body calyx after learning to associate an olfactory stimulus with a visual stimulus (Lent et al., 2007). In our recent report, we demonstrated that CaMKII inhibitors impair the olfactory LTM formation in honey bees (Matsumoto et al., 2014). Are these roles of CaMKII in olfactory memory processing introduced above also true for crickets? In crickets, a CaMKII inhibitor (KN-62 or KN-93) fully impaired induction of LTM, but not STM, paired with multiple-trial conditioning. Moreover, KN-62 fully impaired induction of LTM by a Ca2<sup>+</sup> ionophore (A23187) paired with single-trial conditioning, but not that by a cAMP analog, indicating that CaMKII works upstream of AC for LTM formation cascade. Because KN-62 did not impair LTM induced by a cAMP analog, it was rather surprising to find out that KN-62 or KN-93 inhibits LTM induction with folskolin, an AC activator. The best working theory to explain these observations is that there is an interaction between CaMKII and

Ca2+/CaM, CaMKII and then adenylyl cyclase (AC)-cAMP-PKA signaling. This in turn activates cAMP-responsive element-binding protein (CREB), which results in transcription and translation of genes that are necessary for achieving long-term plasticity of synaptic connection upon other neurons that underlies LTM. NOS, NO synthase; sGC, soluble guanylyl cyclase; Arg, arginine; Gs, Gq, receptor (R)-coupled G-protein; OA, octopamine; ACh, acetylcholine; nAChR, nicotinic acetylcholine receptor; mAChR, muscarinic acetylcholine receptor; PLC, phospholipase C; IP3, inositol 1,4,5-triphosphate; RyR, ryanodine receptor; ER, endoplasmic reticulum.

AC, conceivably through formation of macromolecular complex in a similar manner demonstrated in mammalian CaMKII (Coultrap and Bayer, 2012; Lisman et al., 2012), and when KN-62 or KN-93 binds to CaMKII, AC activation by forskolin may be impaired.

# A Model of the Signaling Pathways for LTM Formation

A putative model of the signaling pathways for olfactory LTM formation in crickets is shown in **Figure 1**, updated from our previous model (Mizunami et al., 2014). The new model illustrates the simplest of all the signaling pathways that account for the results summarized in **Table 1**, which describes the outcomes of co-injection experiments. The following documented findings in several insects are incorporated in this model: (1) in vitro alpha-bungarotoxin (BGT)-sensitive nicotinic acetylcholine receptors (nAChRs) are able to trigger NO synthesis in Kenyon cells of insects (Bicker et al., 1996; Zayas et al., 2002), (2) NO production by NO synthase is stimulated by Ca2+/CaM in Drosophila (Regulski and Tully, 1995), (3) in vitro muscarinic acetylcholine receptors (mAChR) activate CaM by calcium release from the endoplasmic reticulum (ER) via PLC/IP<sup>3</sup> signaling (Hasebe and Yoshino, 2016), (4) calcium release via ryanodine receptors (RyRs) on the ER induces LTM in crickets (Sugimachi et al., 2016), (5) AC is activated by either the G-protein coupled receptor or Ca2+/CaM in Drosophila (Livingstone et al., 1984) and (6) PKA activates CREB which leads to LTM formation in Drosophila (Yin et al., 1995).

Anatomical studies of NO-generating neurons and NOreceptive neurons have been performed in some insects. Putative NO synthase have been revealed histochemically in some neurons of the mushroom body and the antennal lobe, a primary olfactory center, in honey bees (Bicker, 2001), locusts (Müller and Bicker, 1994) and cockroaches (Ott and Elphick, 2002), while immunoreactivity to NO-induced cGMP has been observed in other neurons of the same centers (Bicker et al., 1996; Bicker, 2001). To determine the brain region of NO-generating neurons and NO-receptive neurons in crickets, we investigated the expression patterns of the NOS gene and SGCβ gene by whole-mount in situ hybridization (Takahashi et al., 2009). The SGCβ gene is coding the β subunit of sGC. We observed a high expression level of NOS mRNA in outer Keyon cells of the mushroom body, but not in inner Kenyon cells, in addition to several somata around the antennal lobe and at the base of the visual center optic lobe. On the other hand, we observed a significant level of expression of sGC mRNA in inner Keyon cells. Therefore, NO production is presumed to take place in outer Kenyon cells, and NO permeates into nearby inner Kenyon cells.

One of our next steps is to clarify whether several biological molecules depicted in **Figure 2** indeed contribute to LTM formation in crickets using both pharmacological study and RNAi. The target molecules include nAChR, mAChR, PLC, IP<sup>3</sup> and CREB, which have not been shown to be involved in cricket LTM formation. There are several LTM-related signaling pathways other than those mentioned in this review in other animals, such as N-methyl-D-aspartic acid (NMDA) receptor signaling (Giese et al., 2015; Wang and Peng, 2016), insulin receptor signaling (Zhao and Alkon, 2001; Zhao et al., 2004; Dou et al., 2005; Chambers et al., 2015; Kojoma et al., 2015), mitogenactivated protein kinase (MAPK) signaling (Alfieri et al., 2011; Philips et al., 2013; Shobe et al., 2016), and mechanistic target of rapamycin (mTOR) signaling (Bekinschtein et al., 2007; Blundell et al., 2008; Huang et al., 2013; Buffington et al., 2014; Hylin et al., 2018). Whether these signaling pathways are related to LTM formation in crickets is another issue.

We have established conditioning procedures for different sensory modalities for crickets: olfactory conditioning, visualpattern conditioning and color-vision conditioning. Each conditioning can be classified into two categories: appetitive conditioning and aversive conditioning. Thus, we can examine whether the finding of biochemical cascades in olfactory appetitive learning is applicable to other learning paradigms. For example, in appetitive visual LTM formation, we have shown that NO-cGMP signaling works upstream of cAMP signaling (Matsumoto et al., 2013b). We have also shown that at least NOcGMP signaling participates in aversive visual LTM formation (Matsumoto et al., 2013b). Thus, we conclude that signaling cascades for LTM formation is shared between olfactory and visual learning.

# CONCLUSION

In this review, we overviewed the biochemical cascades for LTM formation based on the results of co-injection experiments with different combinations of LTM-inducing drugs for 'gain of function' and LTM-inhibiting drugs for 'loss of function.' From our pharmacological behavioral studies, we proposed an updated model in which multiple-trial conditioning triggers the NO-cGMP signaling that activates the downstream cAMP signaling through the CNG channel, Ca2+/CaM and CaMKII, leading to the formation of protein synthesis-dependent LTM. A number of molecular actors involved in LTM formation in crickets, such as NOS, NO, cGMP, cAMP, PKA and CaMKII, are known to be involved in mammalian LTM formation. Thus, we conclude that insects, with relatively simple brain structures and neural circuitry, will also be beneficial in exploratory experiments to predict the molecular mechanisms underlying cognitive functions and memory formation in mammals.

# AUTHOR CONTRIBUTIONS

YM, CM, and MM wrote the manuscript and approved the final version.

# FUNDING

This study was supported by Grants-in-Aid for Scientific Research from the Ministry of Education, Science, Culture, Sports and Technology of Japan to YM (Grant No. 16K07434) and to MM (Grant Nos. 16H04814 and 16K18586).

# REFERENCES

fpsyg-09-01014 June 20, 2018 Time: 18:30 # 7


as intracellular messenger in the brain. Nature 336, 385–388. doi: 10.1038/ 336385a0


acquisition studied in olfactory conditioning of maxillary palpi extension response in crickets. Front. Behav. Neurosci. 9:230. doi: 10.3389/fnbeh.2015. 00230


memory consolidation. Eur. J. Pharmacol. 436, 83–87. doi: 10.1016/S0014- 2999(01)01614-4


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

The reviewer MV and handling Editor declared their shared affiliation.

Copyright © 2018 Matsumoto, Matsumoto and Mizunami. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Learning Spatial Aversion Is Sensory-Specific in the Hematophagous Insect Rhodnius prolixus

Sebastian Minoli1,2 \*, Agustina Cano1,2, Gina Pontes1,2, Amorina Magallanes1,2 , Nahuel Roldán1,2 and Romina B. Barrozo1,2 \*

<sup>1</sup> Laboratorio de Fisiología de Insectos, Instituto de Biodiversidad y Biología Experimental y Aplicada-CONICET, Buenos Aires, Argentina, <sup>2</sup> Departamento de Biodiversidad y Biología Experimental-FCEN, Universidad de Buenos Aires, Buenos Aires, Argentina

#### Edited by:

Martin Giurfa, UMR5169 Centre de Recherches sur la Cognition Animale (CRCA), University Paul Sabatier of Toulouse, France

#### Reviewed by:

Clement Vinauger, Virginia Tech, United States Maria Eugenia Villar, Fyssen Foundation, France; UMR5169 Centre de Recherches sur la Cognition Animale (CRCA), University Paul Sabatier of Toulouse, France

> Paul Marchal contributed to the review of Maria Eugenia Villar.

#### \*Correspondence:

Sebastian Minoli minoli@bg.fcen.uba.ar Romina B. Barrozo rbarrozo@bg.fcen.uba.ar

#### Specialty section:

This article was submitted to Comparative Psychology, a section of the journal Frontiers in Psychology

Received: 01 March 2018 Accepted: 28 May 2018 Published: 09 July 2018

#### Citation:

Minoli S, Cano A, Pontes G, Magallanes A, Roldán N and Barrozo RB (2018) Learning Spatial Aversion Is Sensory-Specific in the Hematophagous Insect Rhodnius prolixus. Front. Psychol. 9:989. doi: 10.3389/fpsyg.2018.00989 Even though innate behaviors are essential for assuring quick responses to expected stimuli, experience-dependent behavioral plasticity confers an advantage when unexpected conditions arise. As being rigidly responsive to too many stimuli can be biologically expensive, adapting preferences to time-dependent relevant environmental conditions provide a cheaper and wider behavioral reactivity. According to their specific life habits, animals prioritize different sensory modalities to maximize environment exploitation. Besides, when mediating learning processes, the salience of a stimulus usually plays a relevant role in determining the intensity of an association. Then, sensory prioritization might reflect an heterogeneity in the cognitive abilities of an individual. Here, we analyze in the kissing bug Rhodnius prolixus if stimuli from different sensory modalities generate different cognitive capacities under an operant aversive paradigm. In a 2-choice walking arena, by registering the spatial distribution of insects over an experimental arena, we evaluated firstly the innate responses of bugs confronted to mechanical (rough substrate), visual (green light), thermal (32◦C heated plate), hygric (humidified substrate), gustatory (sodium chloride), and olfactory (isobutyric acid) stimuli. In further experimental series bugs were submitted to an aversive operant conditioning by pairing each stimulus with a negative reinforcement. Subsequent tests allowed us to analyze if the innate behaviors were modulated by such previous aversive experience. In our experimental setup mechanical and visual stimuli were neutral, the thermal cue was attractive, and the hygric, gustatory and olfactory ones were innately aversive. After the aversive conditioning, responses to the mechanical, the visual, the hygric and the gustatory stimuli were modulated while responses to the thermal and the olfactory stimuli remained rigid. We present evidences that the spatial learning capacities of R. prolixus are dependent on the sensory modality of the conditioned stimulus, regardless their innate valence (i.e., neutral, attractive, or aversive). These differences might be given by the biological relevance of the stimuli and/or by evolutionary aspects of the life traits of this hematophagous insect.

#### Keywords: learning, sensory modalities, insects, triatomines, operant, aversive

**Abbreviations:** DCM, dichloromethane; IsobAc, isobutyric acid; NaCl, sodium chloride; PI, preference index.

# INTRODUCTION

fpsyg-09-00989 July 6, 2018 Time: 13:59 # 2

As it happens in most animals, insects' sensory systems can detect a wide range of stimuli but respond only to a few of them, usually the most relevant ones. This process of filtering irrelevant information is essential for any living being, which would otherwise be engaged in a continuous outcome of triggered behaviors belonging to different contexts. Moreover, according to their specific life habits, animals can prioritize the use of different sensory modalities to maximize the exploitation of available resources from the environment. For example, to find a food source some animals use mainly the visual system, while others make use mainly of their chemical senses (i.e., olfactory or gustatory). As a result, stimuli from different modality can be more or less significant for an individual. Usually these differences are reflected in the complexity of particular sensory structures of each species, which sometimes present remarkable specializations of associated sensory organs.

Besides, the set of stimuli to which an organism responds can change along its lifetime, and thus the same individual can stop responding to some and start responding to originally neutral stimuli. This behavioral plasticity can be induced by several factors, such as the nutritional and reproductive status, time of the day, previous experiences, among other. In anyway, regardless its physiological origin, behavioral plasticity allows animals to maximize the efficiency of exploitation of unstable and/or unpredicted environments by allowing animals to modulate their responses according to immediate needs.

In particular, experience dependent plasticity allows animals to finely tune innate responses and even to respond to stimuli that being originally neutral gain certain relevancy after a reinforced experience. It is not surprising then that learning capacity has been revealed in almost all studied animals. In this sense, adapting preferences to time-dependent relevant environmental conditions provide a wider and cheaper behavioral reactivity. Learning involves a complex series of processes that promote reversible modifications in particular behaviors which can be highly adaptive, generating a memory of that event. Two main types of learning have been well described so far: non-associative and associative. The first one is generated after the repetition of a unique type of stimulus that, without any reinforcement increases (sensitization) or decreases (habituation) the intensity and/or frequency of the subsequent response of the individual to the same stimulus (Kandel, 1991; Rakitin et al., 1991; Menzel, 1999). The second is the process by which an association between two stimuli or a behavior and a stimulus is formed, if properly reinforced (Bitterman et al., 1983; Menzel and Muller, 1996). Two main forms of associative learning have been described in animals. In Pavlov's classical conditioning (Pavlov, 1929) a previously neutral stimulus is repeatedly presented together with a reflex-eliciting stimuli followed by a reinforcement, until eventually the neutral stimulus will elicit a response on its own. In Skinner's operant conditioning (Skinner, 1937) a certain behavior is followed by a reinforcement, resulting in an altered probability that the behavior will happen again. Associative forms of learning allow individuals to anticipate events by recognizing marks previously related to them. In this work we designed and applied an associative operant aversive conditioning.

Learning abilities can largely differ across species, individuals and even throughout lifespan and can be modulated by several features of the training procedures (Menzel et al., 2001; Deisig et al., 2007; Giurfa and Sandoz, 2012; Giurfa, 2015). Among them, the salience of the conditioned stimulus has a relevant role in determining the intensity of an association (Menzel and Muller, 1996). As a generality, salient stimuli are more prone to generate a conditioned response than those that do not differ much from the environmental basal sensory information. Thus, given that animals prioritize different sensory modalities according to their habits (e.g., diurnal animals usually make use of visual cues while nocturnal ones do not), the cognitive abilities of an individual might be reflected in these differences. We analyze in this work if stimuli from different sensory modality can generate different cognitive performances in kissing bugs.

Rhodnius prolixus (Heteroptera: Reduviidae: Triatominae) is an hematophagous insect, vector of the Chagas disease in Latin America. Up to date, there are no vaccines that can prevent the transmission of the Trypanosoma cruzi, parasite responsible for this illness in humans. This fact intensifies the relevance of studying this vector of a human disease, as adding knowledge about its physiology, behavior and/or ecology permits to increase the general knowledge about this species and at the same time can help in improving the efficiency of field control strategies. In fact, since Wigglesworth and Gillett's (1934) pioneer works this blood-sucking bug has classically been an experimental model in the study of physiology of behavior in insects. However, it was only few years ago that their learning capacities have captured the attention of researchers. Vinauger et al. (2011a,b) applied a classical conditioning approach and succeeded in training R. prolixus to associate lactic acid (a neutral odor) with food (i.e., positive reinforcement) or with a mechanical perturbation (i.e., negative reinforcement). They found that in further tests, R. prolixus walked toward or against the lactic acid, respectively. Moreover, even if R. prolixus did not present a preference in a walking olfactometer when odors from a live rat and quail were presented simultaneously at opposite sides, an aversive conditioning generated an aversion to one or the other host according to the training procedure (Vinauger et al., 2012). In addition, kissing bugs extend their proboscis (PER, for proboscis extension response) when they perceive a warm object at the correct temperature and distance. Taking advantage of this unconditioned response, Vinauger et al. (2013) demonstrated that the PER of R. prolixus can be modulated by non-associative and associative learning forms. In a completely different context, Minoli et al. (2013) showed that the innate escape response of kissing bugs to the alarm pheromone can be widely modulated by associative and nonassociative conditioning protocols. Moreover, it was reported for the same species that a brief pre-exposure to bitter compounds prevents insects from feeding on an appetitive solution (Pontes et al., 2014). Later, triatomines' cognitive abilities showed to follow a circadian rhythm (Vinauger and Lazzari, 2015). These authors describe that bugs perform well during the night, but

not during the day. Studying the repellent effect of new nontoxic molecules for R. prolixus, Asparch et al. (2016) found that bugs are innately repelled by different bitter molecules, and that this repellence can be modulated by associative and non-associative forms of learning. Indeed, after an aversive operant conditioning, bugs' behavior changed from avoidance to indifference or even to preference, according with the protocol applied (Asparch et al., 2016). In another work, Mengoni et al. (2017) studied the experience-dependent plasticity of the innate attractive response of kissing bugs to feces. These authors describe that after pre-exposing bugs to feces for 24 h, insects were no longer attracted to feces. Finally, by pairing the presence of feces with an aversive mechanical disturbance, nymphs switched from attraction to avoidance of feces.

In this work we addressed the question whether stimuli from different sensory modality can generate different learning performances under an operant aversive protocol. We firstly studied the innate responses of R. prolixus to mechanical, visual, thermal, hygric, gustatory, and olfactory stimuli. Then we analyzed if such responses can be modulated by an operant aversive conditioning. Stimuli could be innately neutral, attractive or aversive and change their perceptual value after training. We discuss possible roles of the modality and/or type of stimulus in the efficiency of the learning process and its relation with the biological relevance of the stimulus.

#### MATERIALS AND METHODS

#### Insects

R. prolixus was reared in an insectary at 28 ± 1 ◦C, 40 ± 10% relative humidity and an L:D 12:12 h inverted photoperiod cycle. Each week, newly emerged fifth instar nymphs were collected from the rearing chamber and maintained unfed for 7–15 days prior to their use in experiments. This is a moderate starving status since once fed, these hematophagous insects can resist up to 60 days without feeding again. Insects were used only once and then discarded. A total of 540 insects were used along this work. All experiments were carried out in functional darkness during the first 6 h of their scotophase (i.e., 0–6 h after lights were turned-off) as to match the maximal activity period described for triatomines (Lazzari, 1992) and at the same time to exclude external visual cues. The temperature of the experimental room was set to 25 ± 1 ◦C before the beginning of each assay and the relative humidity ranged between 30 and 60%.

In order to minimize potential effects of inbreeding, our insectary is frequently provided with new insects by the Servicio Nacional de Chagas (Santa María de Punilla, Córdoba, Argentina). All animals were handled according to the biosafety rules of the Hygiene and Safety Service of the Universidad de Buenos Aires.

#### Two-Choice Walking Arena

To study the responses of R. prolixus to stimuli of different modality, insects were individually released at the center of a walking rectangular acrylic arena of 8 cm × 4 cm, virtually

zone: i.e., the rough side (black triangle, p < 0.05). During test the smooth side was still preferred (black circle, p < 0.05) evincing an experience-dependent behavioral plasticity. Each point represents the mean (±SE) spatial preference of 30 insects individually released in a 2-choice rectangular walking arena. Asterisks denote statistical differences between the PI and the value 0 (p < 0.05) evinced by One sample T-test. Gray shadows show the zone in which punishment was delivered during unpaired and paired trainings.

divided by a line in two equal zones of 4 cm × 4 cm (see inset of **Figures 1**–**6**). According to the experimental series, a particular stimulus was added at one zone of the arena while the opposite zone was maintained as the corresponding control zone. To facilitate the walking behavior of bugs, the floor of the arena was covered with filter paper, which was replaced between replicates to avoid chemical contamination among assays. To avoid spatial heterogeneities other than those intentionally added, the position of the stimuli was switched between left and right side in a pseudorandom manner (i.e., 15 times at each side).

Stimuli from different modalities were tested, i.e., mechanical, visual, thermal, hygric, gustatory, and olfactory. In all cases, once the stimuli were settled and stabilized over the arena, one insect was gently released at its center and left to freely walk during 4 min. During this experimental time, its spatial distribution in relation to the position of the stimulus (e.g., attraction, repellence or indifference) was recorded in video using an infrared-sensitive video-camera connected to a digital recorder. The time spent at each zone of the arena was then obtained from the video films and a preference index (PI) was calculated for each individual as the difference between the time spent at the stimulus zone (Ts) minus the time spent at the control zone (Tc) divided by the total experimental time:

$$PI = \frac{\text{TS} - \text{TC}}{\text{TS} + \text{TC}}$$

PIs near 0 indicate lack of preference (neutral stimulus); PIs close to −1 show preference for the control zone (repellent stimulus); PIs close to 1 show preference for the stimulus zone (attractive stimulus).

punishment was delivered during unpaired and paired trainings.

# Stimuli and Modalities

Responses of R. prolixus to six stimuli of different modality were analyzed in the two-choice walking arena. Although in each case the addition of the stimulus and its corresponding control was accomplished differently, the goal was always the same: generate a spatial heterogeneity in the arena for R. prolixus. We then analyzed if such spatial heterogeneity evoked an innate response in insects and if such responses could be modulated by a previous experience.

#### Mechanical Stimulus

A mechanical stimulus was added in the arena by making multiple holes (∼= 1 mm diameter, ∼= 1 mm distance between holes) with an awl to the filter paper covering the floor of the stimulus zone (inset **Figure 1**). Previous experiments performed in our laboratory show that R. prolixus can detect this roughness in the substrate during walking (unpublished data). The control zone was maintained intact generating a "smooth/rough" spatial heterogeneity.

#### Visual Stimulus

A green led (5 mm diameter, 2.4 V, 520–550 nm) controlled with a dimmer was added outside the arena, 2 cm away from the distal wall of the stimulus zone (inset **Figure 2**). Light could

FIGURE 3 | Responses of R. prolixus to a thermal stimulus. A 32◦C heated plate was attractive for naïve insects (white circle, p < 0.05). Unpaired yoke controls presented the same pattern: i.e., attraction to heat (gray triangle and circle, p < 0.05 in both cases). During training, insects avoided the punished zone: i.e., the hot side (black triangle, p < 0.05). During test insects continued to prefer the hot side (black circle, p < 0.05), evincing that this attraction was not modulated by the aversive conditioning. Each point represents the mean (±SE) spatial preference of 30 insects individually released in a 2-choice rectangular walking arena. Asterisks denote statistical differences between the PI and the value 0 (p < 0.05) evinced by One sample T-test. Gray shadows show the zone in which punishment was delivered during unpaired and paired trainings.

FIGURE 4 | Responses of R. prolixus to a hygric stimulus. Insects innately avoided the humid zone of the arena (white circle, p < 0.05). Unpaired yoke controls presented the same pattern: i.e., hygric avoidance (gray triangle and circle, p < 0.05 in both cases). However, this avoidance disappeared during (black triangle, p < 0.05) and after (black circle, p > 0.05) training, evincing a partial modulation of this avoidance. Each point represents the mean (±SE) spatial preference of 30 insects individually released in a 2-choice rectangular walking arena. Asterisks denote statistical differences between the PI and the value 0 (p < 0.05) evinced by One sample T-test. Gray shadows show the zone in which punishment was delivered during unpaired and paired trainings.

pass through the transparent acrylic wall of the arena and reach the position of insects. Previous works show that R. prolixus can perceive green light (Reisenman et al., 2000). The low intensity

FIGURE 5 | Responses of R. prolixus to a gustatory stimulus. Insects were innately repelled by NaCl (white circle, p < 0.05). Yoke controls presented the same pattern (gray triangle and circle, p < 0.05 in both cases). During training, insects avoided the punished zone: i.e., the H2O side (black triangle, p < 0.05). During test NaCl side was still preferred (black circle, p < 0.05) evincing an experience-dependent behavioral plasticity. Each point represents the mean (±SE) spatial preference of 30 insects individually released in a 2-choice rectangular walking arena. Asterisks denote statistical differences between the PI and the value 0 (p < 0.05) evinced by One sample T-test. Gray shadows show the zone in which punishment was delivered during unpaired and paired trainings.

FIGURE 6 | Responses of R. prolixus to an olfactory stimulus. IsobAc was repellent for these insects (white circle, p < 0.05). Yoke controls presented the same pattern (gray triangle and circle, p < 0.05 in both cases). During training, insects avoided the punished zone: i.e., the DCM side (black triangle, p < 0.05). During test insects continued to avoid the IsobAc side (black circle, p < 0.05), evincing that the repellence could not be modulated by the aversive conditioning. Each point represents the mean (±SE) spatial preference of 30 insects individually released in a 2-choice rectangular walking arena. Asterisks denote statistical differences between the PI and the value 0 (p < 0.05) evinced by One sample T-test. Gray shadows show the zone in which punishment was delivered during unpaired and paired trainings.

chosen (1 ± 0.2 lux) allowed us to offer a punctual visual cue that barely illuminated the arena. No light was added at control zone.

#### Thermal Stimulus

A thermal heterogeneity was generated in the arena by heating the wall at the end of the stimulus zone by contacting it externally with a thermostatized heated plate (inset **Figure 3**). A layer of thermal grease was added between both surfaces to improve thermal conduction. In this way, temperature in the inner side of the acrylic wall of the stimulus zone was stabilized at 32 ± 0.5◦C, while the inner wall of the control zone was maintained at ambient temperature, i.e., 24 ± 0.5◦C. Temperature was chosen as to match skin temperature of triatomines natural hosts.

#### Hygric Stimulus

To generate an hygric heterogeneity in the arena we added 100 µl of distilled water on the filter paper covering the stimulus zone. A micropipette allowed us to distribute the water homogeneously (inset **Figure 4**). Volume added was chosen as to make sure filter paper was wet but did not present puddles. In this way, this zone of the arena was humid, while the control zone was maintained dry. The experiment started immediately after loading the water (i.e., 1 min approximately) in order to minimize water evaporation.

#### Gustatory Stimulus

To generate a gustatory heterogeneity in the arena we added (homogeneously with a micropipette) 100 µl of 1 M NaCl over the filter paper covering the floor of the stimulus zone and 100 µl of distilled water on the control zone (inset **Figure 5**). The NaCl (purchased in Biopack, Argentina) solution was prepared in distilled water. This concentration was chosen as in previous works it was efficient in deterring feeding in the same species (unpublished data). The experiment started immediately after loading the water and the NaCl solution (i.e., 1 min approximately) in order to minimize water evaporation.

#### Olfactory Stimulus

An olfactory gradient was generated over the arena by adding IsobAc at the stimulus zone and DCM, (solvent used to dilute IsobAc) at the control zone. Previous works show that this odorant generates an escape response in this species (Cruz-López et al., 2001; Minoli et al., 2013). The experimental arena was slightly adapted for this series by performing five holes (1 mm diameter) at the bottom of the distal walls of each zone (inset **Figure 6**). Outside these walls, a chamber containing IsobAc communicated with the interior of the arena via the holes. The addition of the odorant was achieved by placing a piece of filter paper (2 × 1 cm) loaded with 1000 µg of IsobAc dissolved in 20 µl of DCM in one chamber and another paper loaded with 20 µl of DCM in the opposite chamber. In this way, vapors released by the papers entered the arena through the holes and generated a chemical gradient. IsobAc and DCM were purchased from Sigma-Aldrich (St. Louis, MO, United States).

#### Operant Aversive Conditioning

To analyze if the responses of R. prolixus to different stimuli are differentially modulated by a previous experience, we applied an operant aversive conditioning and analyze if the insects' innate preferences were modulated or not. For this purpose we used the

TABLE 1 | Stimuli associated with the safe and the punishment side of the arena in each experimental series.


same experimental arena described above but with the addition of a vortex mixer (40 Hz) that, being in contact with the base of the arena, allowed us to generate a vibration that reached the insects via the substrate and was applied as negative reinforcement. This vibration was shown to be innately aversive for R. prolixus (Minoli et al., 2013) and could be voluntarily controlled by the manipulator via a manual switch. Before experiments and for each experimental series we predefined if the stimulus zone or the control zone were associated with the negative reinforcement according to the innate responses of the insects (see **Table 1**). Innately attractive stimuli were positioned at the punishment side and aversive ones at the safe side. This decision was assumed as to intend to turn over the innate valence of the stimulus with the aversive conditioning. Neutral stimuli were arbitrarily placed at the punishment side.

In this way, for each sensory modality, a 4 min training period was applied in which the negative reinforcement was applied to the insects whenever they entered the predefined punishment zone. The vibration ended as soon as the insect entered the safe zone of the arena. Yoke control series were run in parallel in which each individual received the negative reinforcement independently from its position in the experimental arena. The timing, frequency and duration of the vibration were copied from the previously conditioned insect.

The behavior of each individual during training time was registered in video and the individual PIs were computed. Once training ended, the bug was removed from the arena and released in an individual flask for 1 min. Following this time, it was transferred to the two-choice arena where its preference was tested as explain in Section "Two-Choice Walking Arena." Note that separated PIs were registered for training (triangles in the figures) and test (circles).

#### Data Analysis and Statistics

The PI of each individual was computed. For each experimental series (i.e., mechanical, visual, thermal, hygric, gustatory, and olfactory series), thirty individuals were tested in each group (i.e., 30 naïve, 30 yoke control and 30 paired), totalizing 540 insects. Insects were used only once and then discarded. The mean PI of each series was compared against the expected value if there were no preferences, i.e., "0." One-sample T-tests were applied to statistically assess this difference (Sokal and Rohlf, 1995). Normality and homoscedasticity of data were checked. All figures represent the mean PIs (x-axis) and standard errors, and the stimuli presented at each zone of the arena (y-axis).

## RESULTS

Innate responses of R. prolixus in the 2-choice walking arena varied among stimuli (see **Figures 1–6**, white circles). The rough substrate (mechanical stimulus) and the green led (visual stimulus) were neutral, i.e., they did not modify bugs' distribution over the arena. As expected, the heated plate (thermal stimulus) generated an innate attraction. Conversely, the distilled water (hygric stimulus), the NaCl (gustatory stimulus) and the IsobAc (olfactory stimulus) were innately repellent. The experiencedependent plasticity of the responses of R. prolixus varied according to the stimulus and is dissected in the next section.

#### Innate Responses and Experience-Dependent Modulation Lack of Response to a Textured Substrate

Kissing bugs are thigmotactic animals, i.e., they try to maintain physical contact with objects that provide a mechanical stimulus. In their natural environments they remain a great part of the day in contact with different materials from their shelters and with conspecifics. In our experiments, the addition of a rough substrate in the experimental arena did not generate a preference in bugs (**Figure 1**, white circle, p > 0.05). However, the vibration caused by the vortex mixer was clearly perceived as a negative stimulus for bugs, as during training they avoided the punishment zone (**Figure 1**, black triangle, p < 0.05), i.e., the rough substrate. During the test, in which negative reinforcements were no longer delivered, this avoidance for the zone containing the rough substrate was still expressed (**Figure 1**, black circle, p < 0.05), demonstrating that bugs established an association between the physical properties of the substrate and the occurrence of a punishment.

#### Lack of Response to a Punctual Green Light

Previous works show that kissing bugs avoid ambient light (Reisenman and Lazzari, 2006) but can be attracted to low intensity punctual light sources (Minoli and Lazzari, 2006). In the present work, a punctual green light source at one side of the arena was neither attractive nor repulsive for R. prolixus (**Figure 2**, white circle, p > 0.05). Like in the previous series, the negative reinforcement applied at the green led zone caused a spatial preference for the opposite side of the arena (**Figure 2**, black triangle, p < 0.05). During the posterior test, insects continued to avoid the green led zone, demonstrating that insects could associate the visual stimulus with the punishment (**Figure 2**, black circle, p < 0.05).

#### Attraction to Heat

Thermal stimulation is among the most informative cues used by hematophagous insects to find a host (Lazzari and Nuñez, 1989; Lazzari, 2009). In our experimental setup, the addition of a hot plate at one side of the arena produced the highest attraction response registered in this work (**Figure 3**, white circle, p < 0.05). However, when the vibration was applied at the hot zone, bugs avoided it, evincing that the negative value of the vibration is somehow more intense than the positive value of the heat per se (**Figure 3**, black triangle, p < 0.05). However, in this case, differently from previous series, during the posterior test bugs preferred to occupy the heated side just as naïve insects, indicating that the association between heat and the vibration could not be established or was not expressed (**Figure 3**, black circle, p < 0.05).

#### Avoidance of a Wet Substrate

fpsyg-09-00989 July 6, 2018 Time: 13:59 # 7

The presence of distilled water over the walking substrate produced an avoidance behavior in bugs (**Figure 4**, white circle, p < 0.05). Previous studies have shown that kissing bugs present marked humidity preferences (Roca and Lazzari, 1994; Lorenzo and Lazzari, 1998; Guarneri et al., 2002). In this work we show for the first time the existence of an aversion for wet substrates in kissing bugs. Surprisingly, during the conditioning period in which the dry zone of the arena was defined as the punishment zone, insects spent half of the time at each zone (**Figure 4**, black triangle, p > 0.05). This was the only series along this work in which the punishment zone was not avoided during conditioning, suggesting that bugs perceived the wet substrate as negative as the negative reinforcement. However, during the posterior test bugs continued to exhibit a random distribution (**Figure 4**, black circle, p > 0.05), evincing at least a partial modulation of the innate behavior of avoiding wet substrates.

#### Salt Repellence

Once a kissing bug reaches the skin of a potential host, their gustatory sense starts to play a relevant role in its feeding decision. Previous works show that R. prolixus can identify aversive and/or appetitive molecules that will deter or induce the feeding process (Pontes et al., 2014, 2017). We show here that R. prolixus avoids walking in zones containing high concentrations of NaCl (**Figure 5**, white circle, p < 0.05). Just as in previous series (except in the hygric one), during training bugs avoided the punishment zone even if they had to remain in the aversive zone (**Figure 5**, black triangle, p < 0.05). In the posterior test, insects continued to avoid the punishment zone, even if vibrations were no longer delivered, preferring to stay at the NaCl zone (**Figure 5**, black circle, p < 0.05). This result shows an experience-dependent modulation of their gustatory preference.

#### Avoidance of the Alarm Pheromone

Adult kissing bugs release IsobAc as the main component of an alarm pheromone, and nymphs and adults are repelled by this signal (Manrique et al., 2006). In our setup, R. prolixus innately avoided the zone of the arena containing IsobAc (**Figure 6**, white circle, p < 0.05). During training, bugs avoided the punishment side, remaining mostly in the IsobAc zone (**Figure 6**, black triangle, p < 0.05). In the posterior test, bugs avoided the IsobAc (**Figure 6**, black circle, p < 0.05), evincing that they were either not able to generate an association between the olfactory stimulus and the occurrence of the punishment or that they couldn't express it.

#### Yoke Control: Unpaired Delivery of the Negative Reinforcement

In all yoke series, the random delivery of vibration did not affect the expression of the innate behavior of insects. During both, trainings and test, yoke control insects behaved as naïve ones, i.e., a random behavior when confronted to mechanical (**Figure 1**, gray triangle and circle, p > 0.05 in both cases) and visual stimuli (**Figure 2**, gray triangle and circle, p > 0.05 in both cases), an attraction to the heated side of the arena (**Figure 3**, gray triangle and circle, p < 0.05 in both cases) and an aversion for hygric (**Figure 4**, gray triangle and circle, p < 0.05 in both cases), gustatory (**Figure 5**, gray triangle and circle, p < 0.05 in both cases) and olfactory stimuli (**Figure 6**, gray triangle and circle, p < 0.05 in both cases). These results confirm that in the conditioning series presented above, an associative learning was responsible for the modulation observed.

# DISCUSSION

Learning is crucial to maximize the exploitation of resources in unpredictable environments. However, although it is expressed in most animals, it has been widely shown that small changes in the acquisition protocols can drastically modulate the efficiency of learning at different levels. In this work we studied how the sensory modality of the stimuli involved in the conditioning process can be a key factor for the correct acquisition of information from the environment. For this purpose, we maintained a unique operant aversive protocol, being the conditioned stimulus the only parameter that varied between experimental series. We then analyzed and compared if the responses to such stimuli were more or less prone to be modulated by such a previous experience.

Along our experiments, the mechanical vibration showed to be an efficient negative reinforcement for R. prolixus. During training of the two neutral series (i.e., mechanical and visual) insects avoided the punishment zone of the arena, showing that the vibration is indeed perceived by bugs as an aversive stimulus that generates a spatial avoidance (**Figures 1**, **2**, black triangles). In the next four series (i.e., thermal, hygric, gustatory, and olfactory), in which an innate behavior was provoked by the conditioned stimuli, the punishment side was intentionally defined as to match the innately preferred zone of the arena. During the training of the thermal series, insects preferred the not-heated/safe zone of the arena rather than the heated/punishment zone (**Figure 3**, black triangles), evincing that the value of the negative reinforcement was higher than the positive attractive value of the heat. On the other hand, in training phases of innately repellent stimuli insects either lost their innate stimuli avoidance (i.e., hygric, **Figure 4**, black triangle), or they inverted it (i.e., gustatory and olfactory, **Figures 5**, **6**, black triangles), evincing in this case that the negative value of the vibration was higher than that of the aversive stimuli. Being choice experiments, the expression of the spatial preference of the insects for one or the other side of the arena merely reflects a relative preference, and does not allow us to discern if it is the result of an attraction to the preferred side, a repellency for the avoided side, or if of both processes are acting together. Besides, it is worth noting that even if the vibration was clearly perceived as an aversive

stimulus, the avoidance generated during training phases does not imply that the animals are able to modulate their behavior in an associative-dependent manner. For example, vibration was indubitably aversive for kissing bugs during the thermal series training, but during subsequent test they continued to be attracted to heat, suggesting that the association between heat/punishment was either not achieved or could not be expressed.

As expected, some of the stimuli triggered conspicuous innate responses in these insects. Heat, known to be among the most important cues in host finding for R. prolixus (Lazzari, 2009), was attractive for R. prolixus. Contrarily, the addition of IsobAc to the two-choice arena generated an innate repellence. IsobAc is the main component of the alarm pheromone of these insects and is a powerful activator and repellent (Cruz-López et al., 2001; Rojas et al., 2002; Manrique et al., 2006). It is not surprising then that these two intense responses belonging to two different but biologically relevant contexts (i.e., feeding and escaping from danger, respectively) were not modulated by the previous experience. Whereas behavioral plasticity might be a key process in fluctuant environments, innate and rigid responses are probably more adaptive if stable and honest stimuli are involved. In this sense, we can speculate about the possibility that responses to biologically relevant stimuli are less prone to be modulated by a previous experience. In this sense, learning to "not approach" a heat source and/or to stop avoiding the alarm pheromone might result in death by starvation or by being eaten by a potential predator.

Conversely, the innate avoidance of NaCl was effectively modulated by an aversive conditioning. In natural conditions, R. prolixus exerts a chemical scanning of the potential host skin using gustatory receptors present in their antennae. High levels of NaCl over the skin were shown to inhibit feeding of this species (Pontes et al., 2014). Accordingly, our results show that bugs prefer to avoid walking over substrates containing NaCl. However, following conditioning, bugs radically changed their behavior, even preferring to walk over the NaCl-loaded substrate rather than to do it in the control side. This is clear evidence that R. prolixus can learn from their previous experience in an aversive operant paradigm. Compared to the thermal and the olfactory cues discussed in the previous paragraph, learning to stop avoiding salty substrates might not have deleterious consequences.

Even if the visual spectrum and the negative phototaxia of kissing bugs have been quite well studied when ambient light is presented (reviewed in Barrozo et al., 2016), far less is known about the responses of these bugs to dimmed punctual light sources. Light traps have been reported to capture triatomines, but in low quantities (Vazquez-Prokopec et al., 2004; Carbajal de la Fuente et al., 2007). In an indoor flying cage, Minoli and Lazzari (2006) found that adult R. prolixus and Triatoma infestans initiate flight toward a white or an UV light source. Our results show that R. prolixus exhibits a random walking behavior in presence of the green led. However, this lack of response was modulated by the applied aversive conditioning, as insects learned to keep away from the green light to avoid punishment. Similarly, no behavioral preference was registered when different roughness in the substrate was offered to insects. Then, insects started to avoid the rough surface after the conditioning period. Differently from previous series, both the visual and the mechanical cues were neutral for bugs prior to the conditioning. However, animals learned to avoid the negative reinforcement by changing their spatial preference over the arena. Following the previous idea, those behaviors that do not seriously compromise the animal's survival seem to be more easily modulated than those that might do it.

Several studies have shown that ambient humidity plays a relevant role in kissing bugs' distribution (Roca and Lazzari, 1994; Lorenzo and Lazzari, 1999) and host finding (Barrozo et al., 2003). However, no previous data are available about the effect that a humid substrate might have on their walking behavior. We show here for the first time that R. prolixus avoids walking over a wet filter paper. Surprisingly, during conditioning, insects exhibited a random occupancy of each zone of the arena. This is the only series in which bugs did not avoid the punishment zone during training. This result could be attributable to a similarity in the perceived negative value of the vibration (i.e., the negative reinforcement) and the wet substrate. However, the effect of the conditioning became evident during the test, as the innate avoidance of the humid zone vanished, remaining bugs similar amount of time at each zone. This result shows that R. prolixus is able to modulate its hygric avoidance behavior after an aversive operant conditioning, although the intensity of the modulation seems to be low.

Different parameters of a learning protocol can modulate its efficiency (Menzel et al., 2001; Deisig et al., 2007; Giurfa and Sandoz, 2012; Giurfa, 2015). Among them, it has been shown that massed- and spaced-trials conditionings favor short and long term memories, respectively. As well, the intensity of the acquisition process can be modulated by the timing of the contingency between the conditioned stimulus (CS) and the unconditioned stimulus (US). The phase, duration and frequency of the paired presentation of the CS and the US play an important role in the acquisition efficiency. Besides, as a general rule, the higher the salience of a particular stimulus, the better the learning score (Menzel and Muller, 1996). However, these are just few of the relevant factors that can modulate the learning capacity of an individual. In our work, we describe how different stimuli can generate differences in the learning capacities of an animal. Maintaining the same operant protocol (i.e., same training time, same time between training and test, same negative reinforcement, same experimental device, etc) we show that the quality of the stimulus used as CS is a key factor for the efficiency of the learning process. However, being that our experiments were performed under an operant protocol design, the intensity of the negative reinforcement could only be determined by the behavior of each individual (remember that the occurrence of the vibration was determined by the position of each insect in the experimental arena). The number of vibrations (**Supplementary Figure S1A**) and the vibration time (**Supplementary Figure S1B**) varied across experimental series (One-way Anova, p < 0.05 in both cases, statistical differences after Tukey's post hoc comparisons showed in letters in the figures). However, no statistical correlations between any of these

two parameters of training and the learning performances were obtained (**Supplementary Figure S2**, p > 0.05 for all correlations, R 2 showed in the figure). These results suggest that the sensory modality of the conditioned stimulus is the main parameter controlling the efficiency of our aversive conditioning paradigm.

As a general observation, our results allow us to speculate about the possibility that animals can modulate their innate responses more easily if the conditioned response is not directly or indirectly harmful for the individual. In this sense, the attraction toward a heat source is an evolutionary conserved behavior that resisted the conditioning designed and applied during this work. The thermal sense of these bugs is probably the most important input implicated in detecting a potential host (Lazzari and Nuñez, 1989; Ferreira et al., 2007). The inhibition of this behavior would then interfere directly in the feeding process, reason why the modulation of this behavior is probably blocked. In fact, thermal experiments carried out in this work were performed using intermediately starved animals (i.e., 7–15 days). Further experiments using recently fed bugs could help in confirming this hypothesis. Previous studies have shown that the proboscis extension response of R. prolixus in response to a heat source can indeed be negatively modulated by an aversive conditioning (Vinauger et al., 2013; Vinauger and Lazzari, 2015). However, although a priori our results and those presented by Vinauger and collaborators seem to be contradictory, they are instead complementary, as different moments of the feeding process are analyzed in each case. While we registered the approach behavior to the potential food source, Vinauger and collaborators studied the extension of the proboscis to start feeding. It seems then that different phases of the feeding behavior of R. prolixus can be differentially modulated by previous experience. Similarly, the conditioning protocol did not succeed in modulating the avoidance response of these bugs to IsobAc. Adults of this species release an alarm pheromone when a potential danger is near. If kissing bugs were to stop escaping from this cue, their lives would probably be endangered. It is worth noting that we do not claim that R. prolixus is not able to modulate its responses to heat or to IsobAc after a previous experience. Indeed we do not know if modifying one or some of the training protocols can alter this fact. However, being all the protocols identical except for the conditioned stimulus, we conclude that there is at least a difference in the proneness to respond to the different stimuli after a previous experience. In particular, the two stimuli that were not suitable to become predictors of an unpleasant event a priori seem to be the more biologically relevant: heat and alarm pheromone.

Furthermore, the behavioral plasticity observed along this work was not correlated with the innate valence of the stimulus. In this work, R. prolixus was innately repelled by an humid substrate, by NaCl and by IsobAc. However, although the intensity of the avoidance behavior generated by the three aversive cues was quite similar, the experience-dependent modulation of such responses was radically different. On the one side, the innate IsobAc avoidance was not suitable to change with our experimental approach. On the opposite side, the innate NaCl avoidance not only disappeared after the training, but gave place to the expression of a new response: insects preferred the side of the arena containing NaCl. In the middle, the innate avoidance of a humid substrate vanished after training, turning into a random spatial distribution. So, we present here evidences that support the idea that the intensity of the experience-dependent modulation of innate negative responses is strongly dependent on the modality of the conditioned stimulus and not on its innate valence.

On the other hand, the two originally neutral stimuli tested in this work elicited a conditioned response after the aversive conditioning. Neutral stimuli are detected by the sensorial system but do not elicit a particular response. In this case, the texture of the substrate or the presence of a dimmed green light did not produce an innate behavior of kissing bugs. However, after training, insects avoided these two originally neutral cues. These results are aligned with the idea that individuals can modulate their innate responses only if the conditioned response is not directly or indirectly harmful for them, as learning to be attracted to a rough substrate or to a green light do not compromise the animal's survival.

It is worth noting that all experiments were performed in a 2-choice experimental design. In this way, each side of the arena assembled sensory information that guided the insect to a particular spatial preference. For naïve groups, the observed innate preference is the result of the comparison between the valence of the stimuli added at each side. However, during training the negative reinforcement was temporally and spatially coupled with the stimulus added at one side of the arena, for what the final decision of the insect is more complex. Moreover, imagine stimuli "A" and "B," being neutral. Applying a negative reinforcement associated to "A" can generate two possible processes that could induce learning: (1) A−: an inhibitory one generated by the negative experience of walking over a vibratory substrate with stimulus "A," or 2) B+: an excitatory one generated by the positive experience of not receiving the vibration when walking over the substrate with the stimulus "B." In both cases, if learning was to occur, a conditioned avoidance of "A" would be the observed behavioral output. However, the resulting modulation of the innate behavior may arise from the action of one, the other, or the combined action of these two processes. So, a test with an animal spending more time at the "B" side could be due to an acquired repellency to "A," to an acquired attraction to "B," or to the combined action of both phenomena. In any case, even if in our work we cannot dissect the exact mechanism (e.g., A− or B+) involved in the experience-dependent modulation of the innate behavior of R. prolixus, we unequivocally show that this modulation is dependent on the modality of the conditioned stimulus.

Even if the role of many parameters of a training protocol were shown to be relevant, to our knowledge this is the first work in which the sensory modality of the conditioned stimulus is considered as a modulator of learning processes. Results were quite clear to show that the same conditioning protocol applied together with different stimuli as CS can render very different results, going from not being able to modulate a particular response up to radically change the innate preferences of these bugs. This work enriches the knowledge about cognition processes in arthropods, adding new insights about the behavioral

plasticity of an hematophagous insect model. Moreover, taking in consideration that R. prolixus is an insect-vector of a human disease and that its DNA has been recently sequenced, it can become a promising model in the learning and memory field. We believe that the experience-dependent modulation of the behavior of these insects should be taken into consideration at the moment of designing control and monitoring field strategies in endemic regions.

#### AUTHOR CONTRIBUTIONS

SM, AC, GP, AM, and NR carried out the experiments. SM performed the statistical analysis. SM and RB designed and coordinated the study and wrote the manuscript.

#### FUNDING

This work was supported by the Agencia Nacional de Promoción Científica y Tecnológica (ANPCyT, FONCyT grant code: Préstamo BID PICT 2013-1253) and the Consejo Nacional de Investigaciones Científicas y Técnicas (CONICET).

# REFERENCES


### ACKNOWLEDGMENTS

Authors thank to G. P. Jerez Ferreyra for his invaluable help rearing and maintaining the insect colony.

# SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpsyg. 2018.00989/full#supplementary-material

FIGURE S1 | Intensity of the negative reinforcement during the operant conditioning. (A) The number of vibrations received by R. prolixus during trainings (One-way ANOVA, p < 0.05) and (B) the vibration time received by R. prolixus during trainings varied across modalities (One-way ANOVA, p < 0.05). Each column represents the mean (±SE) number of vibrations or the vibration time of 30 insects during trainings in a rectangular walking arena. Different letters denote statistical differences between series evinced by Tukey's comparisons.

FIGURE S2 | Correlations between intensity of the negative reinforcement and learning performances. No statistical correlations were found between the number of vibrations or the vibration time and the learning performances of R. prolixus (p > 0.05 in all cases) of all experimental series. (A) Mechanical series, (B) visual series, (C) thermal series, (D) hygric series, (E) gustatory series, (F) olfactory series.



Reduviidae) in rural northwestern Argentina. J. Med. Entomol. 41, 614–621. doi: 10.1603/0022-2585-41.4.614


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

The reviewer MV and handling Editor declared their shared affiliation.

Copyright © 2018 Minoli, Cano, Pontes, Magallanes, Roldán and Barrozo. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Application of a Prediction Error Theory to Pavlovian Conditioning in an Insect

#### Makoto Mizunami<sup>1</sup> \*, Kanta Terao<sup>2</sup> and Beatriz Alvarez<sup>1</sup>

<sup>1</sup> Faculty of Science, Hokkaido University, Sapporo, Japan, <sup>2</sup> Graduate School of Life Sciences, Hokkaido University, Sapporo, Japan

Elucidation of the conditions in which associative learning occurs is a critical issue in neuroscience and comparative psychology. In Pavlovian conditioning in mammals, it is thought that the discrepancy, or error, between the actual reward and the predicted reward determines whether learning occurs. This theory stems from the finding of Kamin's blocking effect, in which after pairing of a stimulus with an unconditioned stimulus (US), conditioning of a second stimulus is blocked when the two stimuli are presented in compound and paired with the same US. Whether this theory is applicable to any species of invertebrates, however, has remained unknown. We first showed blocking and one-trial blocking of Pavlovian conditioning in the cricket Gryllus bimaculatus, which supported the Rescorla–Wagner model but not attentional theories, the major competitive error-correction learning theories to account for blocking. To match the prediction error theory, a neural circuit model was proposed, and prediction from the model was tested: the results were consistent with the Rescorla–Wagner model but not with the retrieval theory, another competitive theory to account for blocking. The findings suggest that the Rescorla–Wagner model best accounts for Pavlovian conditioning in crickets and that the basic computation rule underlying Pavlovian conditioning in crickets is the same to those suggested in mammals. Moreover, results of pharmacological studies in crickets suggested that octopamine and dopamine mediate prediction error signals in appetitive and aversive conditioning, respectively. This was in contrast to the notion that dopamine mediates appetitive prediction error signals in mammals. The functional significance and evolutionary implications of these findings are discussed.

Keywords: blocking, classical conditioning, cricket, dopamine, error-correction learning, invertebrate, octopamine, Rescorla–Wagner model

# INTRODUCTION

Pavlovian (or classical) conditioning is a form of associative learning found in many vertebrates and invertebrates (Perry et al., 2013) that is fundamental for animals' survival since it allows them for finding suitable food, avoiding toxic food, escaping from predators, and detecting mates. This type of learning occurs when an originally unimportant stimulus (conditioned stimulus, CS) becomes associated with a biologically significant stimulus (unconditioned stimulus, US) such that it induces a response (conditioned response, CR) to the CS thereafter. The error-correction learning rule has

#### Edited by:

Lars Chittka, Queen Mary University of London, United Kingdom

#### Reviewed by:

Martha Escobar, Oakland University, United States Bertram Gerber, Leibniz Institute for Neurobiology, Germany

> \*Correspondence: Makoto Mizunami mizunami@sci.hokudai.ac.jp

#### Specialty section:

This article was submitted to Comparative Psychology, a section of the journal Frontiers in Psychology

Received: 21 March 2018 Accepted: 03 July 2018 Published: 23 July 2018

#### Citation:

Mizunami M, Terao K and Alvarez B (2018) Application of a Prediction Error Theory to Pavlovian Conditioning in an Insect. Front. Psychol. 9:1272. doi: 10.3389/fpsyg.2018.01272

**140**

been thought to account for associative learning in mammals (Pearce, 2008; Mazur, 2013) but little is known about whether the same is true for any species of invertebrates (for earlier attempts in honey bees, see Greggers and Menzel, 1993; Smith, 1997). In this article, we briefly review some basic knowledge of computational rules governing Pavlovian conditioning in both vertebrates and invertebrates and their possible neural substrates, with a special focus on our recent finding that the error correction learning rule seems to best account for Pavlovian conditioning in crickets.

## PREDICTION ERROR THEORIES FOR MAMMALIAN PAVLOVIAN CONDITIONING

In associative learning in mammals, a widely accepted view is that the discrepancy, or error, between the reward an animal gets and the reward that the animal predicts (or expects) determines whether learning occurs (Rescorla and Wagner, 1972; Pearce, 2008; Mazur, 2013). The error-correction theory has been applied to learning since at least in 1950s (Bush and Mosteller, 1951) and developed into a refined form in 1970s to account for the finding of blocking phenomenon by Kamin (1969). Blocking takes place when a stimulus (X) that had been paired with a US blocks the subsequent association of a novel stimulus (Y) in a second training phase in which the novel stimulus is presented in compound with X and reinforced by the same US. After this training, when the response to Y alone is tested, it is typically observed that animals do not respond to this stimulus (but notice also that some researchers like, Maes et al., 2016, reported difficulties in replicating blocking effect in rats). The finding of the blocking effect suggests that the strength of temporal contingency (correlation) between the CS and the US, known as a critical factor for conditioning to occur (Rescorla, 1968), is not the only factor that determines the occurrence of learning. Kamin proposed that "surprise" is necessary for learning, and that learning about a stimulus (Y) is blocked when the US is fully predicted by another stimulus (X). This proposition was later formulated into the Rescorla–Wagner model, the most influential form of the error-correction learning theory (Rescorla and Wagner, 1972), which assumes that the discrepancy between the strength of the actual US and total strengths of the predicted US by all the CSs determines the amount of learning (**Table 1A**). Subsequent studies in mammals suggested that dopamine (DA) neurons in the ventral tegmental area of the midbrain mediate prediction error signals for appetitive US, which provided the basis to investigate neural circuit mechanisms of Pavlovian conditioning (Schultz, 2013; Steinberg et al., 2013).

There are theories other than the Rescorla–Wager model that can account for the blocking effect (Miller et al., 1995; Pearce, 2008; Mazur, 2013). The most influential competitive ones are the attentional theories proposed by Mackintosh (1975) and by Pearce and Hall (1980), which are refined versions of the errorcorrection learning theory and account for blocking by decreased attention to the CS (**Tables 1B,C**). It can be stated that Rescorla– Wagner model focuses on US processing whereas attentional models focus more on CS processing. Another notable theory is the comparator hypothesis (Miller and Matzel, 1988), which accounts for blocking by competition between CSs during the memory retrieval process. Remarkably, although efforts have been directed to experimentally test these different theories, which of the theories mentioned best accounts for computational rules governing Pavlovian conditioning remains unclear in any conditioning system (Miller et al., 1995; Pearce, 2008; Mazur, 2013).

## STUDIES ON NEURAL PROCESSING UNDERLYING PAVLOVIAN CONDITIONING IN INVERTEBRATES

Whether error-correction learning models such as the Rescorla– Wagner model represent computational rules underlying learning in any species of invertebrates remained unknown until recently. One of the reasons for the lack of such study is the difficulty in establishing experimental procedures to convincingly demonstrate blocking. In insects, for example, some earlier studies in honey bees (e.g., Smith, 1997; Hosler and Smith, 2000) showed a blocking-like effect but more recent studies failed to establish blocking as a robust phenomenon in honey bees (Guerrieri et al., 2005; Blaser et al., 2006, 2008). Second, although blocking has been reported in the slug Limax maximus (Sahley et al., 1981), the snail Cornu aspersum (formerly Helix aspersa, Acebes et al., 2009; Prados et al., 2013a) and the planaria Dugesia tigrina (Prados et al., 2013b) no attempts have been made to investigate which computational model best accounts for blocking in any of these invertebrate species.

Many of the previous studies on the neural basis of Pavlovian conditioning in invertebrates focused on clarifying the cellular and molecular mechanisms that allow animals to detect the

TABLE 1 | Error-correction learning theories to account for blocking.


In A, V is associative strength that refers to the strength of the CS-US, which corresponds to US prediction, 1V is the change in V that results from a particular conditioning trial, V<sup>6</sup> is total association strengths of all CSs present in a conditioning trial, λ is the magnitude of the US and reflects the maximum strength of the CS-US association that can be achieved, and α is a learning-rate parameter reflecting the intensity of the CS. The model accounts for blocking by decreased (λ–V6) reflecting a change of V as a result of preceding conditioning trials. In B, α<sup>A</sup> is the amount of attention to CSA, V<sup>X</sup> is the associative strength of all stimuli other than CS<sup>A</sup> present in a given trial. The theory accounts for blocking by decreased α<sup>A</sup> as a result of preceding trials. In C, α<sup>A</sup> n is the amount of attention to CS<sup>A</sup> of the n-th trial, and S<sup>A</sup> is a parameter that depends on intensity of CSA. The model accounts for blocking by decreased αA. Description of equations follows Pearce and Hall (1980).

coincident and correlated occurrence of the CS and the US, a prerequisite for Pavlovian conditioning. In Pavlovian conditioning of gill withdrawal responses in the sea hare Aplysia californica, it has been demonstrated that neural signals mediating CS and US converge in some neurons of the nervous system and that type 1 adenylyl cyclase (AC), which catalyzes ATP to produce cAMP, and the N-methyl-D-aspartate (NMDA) receptor, a type of glutamate receptor, serve as key molecules for the detection of coincident arrival of CS and US signals to these neurons to lead to modification of the efficacy of synaptic transmission that underlies conditioning (Abrams and Kandel, 1988; Hawkins and Byrne, 2015). Similarly, in the fruit-fly Drosophila melanogaster, it has been shown that type 1 AC in intrinsic neurons (Kenyon cells) of the mushroom body, a higher-order associative center in the insect brain (Menzel and Giurfa, 2006; Watanabe et al., 2011; Burke et al., 2012; Liu et al., 2012), serve as key molecules to detect coincident arrival of the olfactory CS and the electric shock or the sucrose US signals to these neurons for achieving conditioning (Davis, 2005; Gervasi et al., 2010). However, whether such coincidence detection mechanisms are sufficient to achieve Pavlovian conditioning in these species remains unclear.

# NEURAL SUBSTRATES UNDERLYING PAVLOVIAN CONDITIONING IN CRICKETS

We recently investigated whether blocking occurs in Pavlovian conditioning in the cricket Gryllus bimaculatus. Crickets are newly emerging experimental animals in which associative learning is explored by pairing visual or olfactory cues with either water (to elicit appetitive learning) or with sodium chloride (to induce aversive learning). With these procedures, the neural mechanisms that are involved in both the acquisition and the retrieval of the CR of Pavlovian conditioning have been investigated in some detail (Matsumoto and Mizunami, 2002; Matsumoto et al., 2006, 2018; Mizunami et al., 2014, 2015; Matsumoto Y. et al., 2016). For example, concerning the acquisition of both olfactory and visual learning, we showed that pharmacological blockade of octopamine (OA)-ergic synaptic transmission impairs appetitive but not aversive Pavlovian conditioning, whereas pharmacological blockade of DA-ergic transmission impairs aversive conditioning but not appetitive conditioning (Unoki et al., 2005, 2006; Mizunami et al., 2009; Nakatani et al., 2009; Matsumoto et al., 2015; Mizunami and Matsumoto, 2017). The results obtained in the pharmacological studies were further confirmed in subsequent studies on the effects of knockout or knockdown of genes that code DA receptors or OA receptors by the CRISPR/cas9 system (Awata et al., 2015) or by RNAi (Awata et al., 2016). These findings suggest that OA neurons and DA neurons mediate neural signals representing appetitive and aversive US, respectively, in both olfactory and visual conditioning. Moreover, OA and DA neurons are also involved in the execution of the CR (or in the retrieval of the memory): blockade of OA-ergic transmission impaired CR execution after appetitive conditioning, but not after aversive conditioning with sodium chloride, and blockade of DA-ergic transmission impaired the execution of the CR after aversive conditioning but not after appetitive conditioning (Mizunami et al., 2009). Therefore, it has been concluded that activation of OA neurons is needed for the execution of a CR after appetitive conditioning, whereas activation of DA neurons is needed for the execution of an aversive CR. These results have been integrated in a neural circuit model for Pavlovian conditioning in crickets, which is assumed to represent neural circuitry of the mushroom body (Mizunami et al., 2009). The model accounted for two higher-order learning phenomena, namely second-order conditioning (Mizunami et al., 2009) and sensory preconditioning (Matsumoto et al., 2013). This model provided the basis to construct a model to account for blocking described in subsequent sections.

Roles of OA and DA in mediating appetitive and aversive signals in Pavlovian learning have also been reported in honey bees (Hammer and Menzel, 1998; Farooqui et al., 2003; Vergoz et al., 2007, but see Perry et al., 2016 for bumblebees). In fruitflies, on the other hand, it has been concluded that different classes of dopamine neurons projecting to the mushroom body mediate appetite and aversive signals (Burke et al., 2012; Liu et al., 2012). It seems that the neurotransmitter mediating appetitive signals differs in different species of insects, although that mediating aversive signals is conserved among insects.

## APPLICABILITY OF PREDICTION ERROR THEORY TO PAVLOVIAN CONDITIONING IN CRICKETS

Experiments showing blocking with crickets were conducted, at first, with an appetitive procedure in which water was used as the US. Crickets were subjected to four conditioning trials in which they were exposed to stimulus X immediately before the presentation of water (X+) and were then subjected to compound trials in which stimulus X was presented together with a new stimulus Y followed by the same US (XY+), X and Y being stimuli of different sensory modalities (an olfactory and a visual pattern stimulus, counterbalanced; Terao et al., 2015). Crickets subjected to this training did not respond to Y. In contrast, control crickets that were exposed to unpaired presentations of X and the US (X/+) and then to paired and reinforced presentations of the compound (XY+) or crickets that received only XY+ training exhibited normal learning of Y. Similar results were also obtained in experiments in which blocking was assessed by means of an aversive conditioning procedure (i.e., NaCl was used as the US; Terao and Mizunami, 2017). The results showed that blocking occurs in both appetitive conditioning and aversive conditioning in crickets.

As already mentioned, the most influential models to account for blocking are the Rescorla-Wagner model (Rescorla and Wagner, 1972), the attentional theories proposed by Mackintosh (1975) and by Pearce and Hall (1980), and the retrieval theory (or comparator hypothesis) proposed by Miller and Matzel (1988). However, whether blocking is better accounted for by any of the mentioned models has not been tested in an invertebrate species, except that Smith (1997) examined blocking

in honey bees and argued that the Rescorla–Wagner model can at least in part account for blocking but the attentional theories seem not to account for it. To discriminate among these models, one-trial appetitive blocking experiments were performed. In such experiments crickets received X+ training trials followed by one single XY+ training trial. We used one compound conditioning trial because the Rescorla–Wagner model predicts that such training will result in blocking of Y, whereas attentional theories do not (Mackintosh, 1975; Pearce and Hall, 1980). Our results showed that crickets that received X+ training followed by one XY+ compound-conditioning trial did not respond to Y. In contrast, control crickets that were exposed to unpaired presentations of X and the US followed by one XY+ compound training trial or that received only one XY+ training trial exhibited normal learning of Y. The results supported the Rescorla–Wagner model but not the attentional theories for appetitive conditioning (Terao et al., 2015). We also investigated whether blocking with one XY+ training trial can be accounted for by assuming simple selective attentional process not coupled to error-correction learning, and the results were not consistent with this possibility (Terao et al., 2015). In the case of aversive conditioning (i.e., using NaCl as the US), however, a blocking experiment with one compound trial could not be performed since previous studies have shown that one aversive X+ conditioning trial does not result in aversive learning (Unoki et al., 2005, 2006). Therefore, discrimination of the Rescorla– Wagner model and attentional theories in aversive conditioning remains to be explored. The possible applicability of the retrieval theory will be discussed in a later section.

To account for these findings, we proposed a neural circuit model of Pavlovian conditioning in crickets that matches the Rescorla–Wagner theory (**Figure 1A**; Terao et al., 2015; Terao and Mizunami, 2017), by revising our previous model (Mizunami et al., 2009). The major assumption in our model is that pairing of the CS and the US lead to the enhancement of synaptic transmission from "CS" neurons to three classes of neurons, i.e., "CR," "OA1/DA1," and "OA2/DA2" neurons, in which "CS" neurons are neurons mediating signals about CS (which may represent intrinsic neurons of the mushroom body) and "CR" are neurons that lead to the CR when they are activated (which may represent output neurons of the mushroom body lobes). "OA1/DA1" or "OA2/DA2" neurons are separate classes of OA or DA neurons that receive signals about appetitive or aversive USs (which may represent OA or DA neurons projecting to the mushroom body lobes). "OA1/DA1" neurons (colored in yellow in **Figure 1A**) govern enhancement of "CS-CR" synapses (but not execution of a CR) whereas "OA2/DA2" neurons govern execution of a CR (but not enhancement of "CS-CR" synapses) and here we focus on the former neurons. The model assumes that "OA1/DA1" neurons are critical for error-correction computation, in that (1) the efficacy of "CS-OA1/DA1" inhibitory synapses increases by coincident activation of "CS" and "OA1/DA1" neurons during CS-US pairing trials, (2) inhibitory inputs to "OA1/DA1" neurons represent signals about US prediction by the CS whereas excitatory inputs to these neurons represent US signals, (3) responses of "OA1/DA1" neurons during CS-US pairing trials, hence, represent US

FIGURE 1 | Neural models of Pavlovian conditioning in crickets proposed by Terao et al. (2015) and Terao and Mizunami (2017). (A) Description of the model that has been revised from the model by Mizunami et al. (2009) to match the prediction error theory. The model assumes two classes of OA and DA neurons. One is "OA1/DA1" neurons (colored in yellow) that govern enhancement of "CS-CR" synapses (but not execution of a CR). The other is "OA2/DA2" neurons that govern execution of a CR or memory retrieval (but not enhancement of "CS-CR" synapses). The model also assumes that (1) "CS" neurons [which may represent intrinsic neurons (Kenyon cells) of the mushroom body] that convey signals for CS make silent or weak synaptic connections with dendrites of "CR" neurons [which may represent efferent (output) neurons of the lobes (output regions) of the mushroom body], activation of which leads to a CR, but these synaptic connections are silent or very weak before conditioning, (2) "OA1/DA1" neurons receive excitatory synapses that represent appetitive/aversive US signals and silent or very weak inhibitory synapses from "CS" neurons before training, which are strengthened by CS-US pairing, (3) during training, "OA1/DA1" neurons receive excitatory synaptic input that represents actual US and inhibitory input from "CS" neurons that represents US prediction by CS, and thus their activities represent US prediction error signals, (4) "OA2/DA2" neurons receive excitatory synapses that represent US signals and silent or very weak excitatory synapses from "CS" neurons before training, which are strengthened by CS-US pairing, and (5) "OA2/DA2" neurons make synaptic connections with axon terminals of "CS" neurons, and coincident activation of "CS" neurons and "OA2/DA2" neurons is needed for activation of "CR" neurons (AND gate) and for production of a conditioned response. Presentation of a CS after CS-US pairing activates "CS" neurons and then "OA2/DA2" neurons and thus activates "CR" neurons to lead to a CR. Synapses for which the efficacy can be changed by conditioning are colored in red and marked as "modifiable." Excitatory synapses are marked as triangles, and inhibitory synapses are marked as bars. UR: unconditioned response. (B) Accounts for blocking by the model. "OA2/DA2" neurons in the model in (A) are not shown in (B) for simplicity. The models are modified from Terao et al. (2015) and Terao and Mizunami (2017) with permission.

prediction error signals, and (4) after sufficient amount of training, responses of "OA1/DA1" neurons during CS-US pairing decrease to the zero level and hence no further enhancement of "CS-CR" synapses occurs. Details of the model are shown in the

legend of **Figure 1A**, and how responses of "OA1/DA1" neurons to paired CS-US presentations represent US prediction error signals is described in **Table 2**. As for models of the mushroom body that are intended to account for some other memory tasks, see literatures such as Peng and Chittka (2017) and Roper et al. (2017).

**Figure 1B** depicts how the model accounts for blocking. CS1-US pairing trials strengthen "CS1-OA1/DA1" inhibitory synapses so that responses of "OA1/DA1" neurons during trials are diminished to the zero level. Therefore, when the CS1-CS2 compound is subsequently presented and reinforced with the same US, "OA1/DA1" neurons produce no responses and hence, no enhancement of "CS2-CR" synapses occur (Terao et al., 2015).

One of the predictions that can be made from the model is that, in the case of appetitive conditioning, blockade of output synapses from OA neurons by administration of an OA receptor antagonist (e.g., epinastine) during the conditioning of a stimulus Y (Y+ training) impairs learning of Y since normal synaptic outputs from "OA1" neurons are needed for enhancement of "CS-CR" synapses. This treatment, however, would not affect the prediction error computation, since synaptic outputs from "OA1" neurons do not participate in prediction error computation (**Figure 1B**; Terao et al., 2015). Therefore, administration of epinastine before Y+ training would still allow for error correction to take place in each trial, even though it prevents an enhancement of "CS-CR" synapses necessary for learning. The model thus predicts that subsequent Y+ training after recovery from the effect of epinastine should produce no learning if the associative strength of the "CS-OA1" synapses reaches the maximum after initial Y+ training. Crickets of the experimental group indeed exhibited no learning of Y. In contrast, crickets in the control group that were administrated with epinastine before unpaired presentation of Y and US and then subjected to Y+ training after recovery from the effect of epinastine exhibited normal learning of Y. We referred to this inhibitory phenomenon as "auto-blocking," because learning of Y seems to be blocked by the prediction of the US by Y itself (and not by another stimulus, X, as in the case of blocking

TABLE 2 | Information coded in the responses of "OA1/DA1" neurons in the model of Figure 1.


Responses of "OA1" or "DA1" neurons in the model shown in Figure 1 to appetitive or aversive US, CS, and paired presentation of CS and US before and after conditioning. These neurons are assumed to govern enhancement of synaptic transmission underlying conditioning. After completion of training, these neurons receive excitatory synaptic input when a US is presented and receive inhibitory synaptic input when a CS is presented, the former representing US signals and the later representing US prediction signals. Paired presentations of CS and US induces no responses to these neurons if US-induced excitatory input was canceled by an inhibitory input induced by a CS. In such situations, no further enhancement of synaptic transmission occurs. USP, US prediction; USPE, US prediction error. Responses are indicated as all or none (1 or 0). <sup>∗</sup>Negative value in the parentheses indicates inhibitory synaptic input. Based on Terao et al. (2015).

experiment) (Terao et al., 2015). The absence of CR in the test could also be explained by the comparator model if memory is formed in the second training but not retrieved in the test due to competition of memories formed in the initial and second trainings. Such competition, however, is difficult to assume since results of all our previous studies suggest that no memory is formed in the first training (e.g., Unoki et al., 2005). Taken together, one-trial blocking and the auto-blocking phenomenon suggest that the Rescorla–Wagner model is the one that best accounts for appetitive conditioning in crickets (Terao et al., 2015). In addition, auto-blocking experiments suggest that OA neurons mediate appetitive prediction error signals.

Subsequent studies also showed auto-blocking in an aversive conditioning experiment. Crickets were first administered with a DA receptor antagonist (flupentixol) before training with Y+ (or before exposure to unpaired presentations of Y and + in the case of the control group). As in the previous case, subsequent Y+ training after animals had recovered from the effect of flupentixol did not result in learning of Y (Terao and Mizunami, 2017), whereas animals in the control group showed an increased aversion to Y. The results suggest that the Rescorla–Wagner model or other forms of error-correction learning theories, but not the retrieval theory, best account for aversive conditioning. The results of auto-blocking experiments also suggest that DA neurons mediate aversive prediction error signals.

It should be noted, however, that we do not suggest that errorcorrection learning theories account for all aspects of Pavlovian conditioning in crickets. The model proposed to account for Pavlovian conditioning in crickets assumes synaptic plasticity in three different synapses in the circuitry and suggests that the plasticity of one type of synapses ("CS-CR" synapses) is governed by US prediction error whereas the plasticity of the other two synapses ("CS-OA1/DA1" and "CS-OA2/DA2" synapses) is governed by coincident occurrence of CS and US. Moreover, we have observed second-order conditioning (Mizunami et al., 2009) in crickets, which is difficult to be accounted for by the Rescorla–Wagner model without appropriate revisions (Miller et al., 1995). We have proposed that these learning phenomena in crickets can be accounted for by neural models that assume no error-correction computation (specifically, by neural pathways involving "OA2/DA2" neurons) (Mizunami et al., 2009; Matsumoto et al., 2013; Terao et al., 2015).

It can be pointed out that major predictions from our model differ from those of the temporal difference (TD) model (Sutton and Barto, 1987), a variant of error-correction learning models and frequently used for simulations of activities of dopamine neurons in the midbrain in primates. It has been shown that those neurons in primates are activated by learned CS and less by predicted US after Pavlovian conditioning, in accordance with the TD model (Schultz, 2015). Interestingly, some of these features have also been found in a ventral unpaired neuron, an OA neuron in the subesophageal ganglion in honey bees that mediates sucrose signals in appetitive olfactory conditioning (Hammer, 1993). In our model, on the other hand, activities representing the US prediction by the CS (i.e., responses to learned CS) and those representing US prediction error (i.e., less responding to predicted US during paired CS-US presentation after training)

are assumed in separate classes of aminergic neurons (i.e., "OA2/DA2" and "OA1/OA1" neurons) for simplification of the model. Physiological investigations are needed to clarify the validity of our model.

## FUNCTIONAL AND EVOLUTIONARY CONSIDERATIONS

The finding that an error-correction learning rule accounts for Pavlovian conditioning in crickets is remarkable since it suggests that the basic computational rules underlying Pavlovian learning in crickets are the same to those in mammals. Error-correction computation, one of fundamental neural computations executed in the mammalian brain, can also be achieved in the small brain of crickets. It is thus of interest to elucidate the neural circuit mechanisms underlying the error-correction learning in crickets, and in other species of invertebrates, to compare them with those in mammals. In mammals, midbrain DA neurons are thought to mediate prediction error signals for appetitive stimuli, and whether DA neurons also mediate aversive prediction error signals is under debate (Schultz, 2013; Matsumoto H. et al., 2016). In mice, it has been suggested that prediction error signals observed in midbrain DA neurons are the result of summation of information across multiple brain areas, rather than prediction error signals being computed in a specific brain area (Tian et al., 2016). In crickets, we hypothesize that OA and DA neurons projecting to the mushroom body mediate appetitive and aversive prediction error signals, respectively (Terao et al., 2015; Terao and Mizunami, 2017). Anatomical and physiological characterizations of these OA and DA neurons should pave the way for elucidating the ubiquity and differences of the neural mechanisms underlying prediction error computation among animals of different phyla.

Some questions arise concerning the functional significance and evolution of the error-correction learning rule underlying Pavlovian conditioning in crickets. An important question is what are the functional advantages of having such associative learning systems in which coincident and correlated occurrence of a CS and a US is not sufficient to lead to learning. To facilitate discussion on this issue, we assume that many of the Pavlovian conditioning systems in invertebrates are based on a simpler learning rule, namely, they are based solely on the detection of coincident or contingent occurrence of a CS and a US, as has been assumed by many neurobiologists. It can be argued that an error-correction learning system is advantageous when multiple CSs occur in association with a US, since, in such a system, the magnitude of learning of a given CS is determined by its relative "surprisingness" or by to what extent the CS predicts the US. This learning system is more efficient in that it prevents learning of redundant cues compared to a learning system that is solely based on the detection of temporal coincidence or contingence, in which all CSs that occur in the same temporal relationship with a US should be equally learned. An error-correction learning, however, should have a cost, in that it requires elaborate neural circuits in the brain, and the development and maintenance of such circuits should be costly. Such a cost, however, is likely to be moderate since it is affordable for crickets that have only small brains.

Another question to be addressed in the future is to what extent the Pavlovian conditioning system with the errorcorrection rule is ubiquitous among invertebrates. The blocking phenomenon, a hallmark for the existence of the error-correction learning rule, has so far been reported only in slugs (Sahley et al., 1981), snails (Acebes et al., 2009; Prados et al., 2013a), and planarians (Prados et al., 2013b) but whether it occurs by errorcorrection learning or by other process, such as cue competition during memory retrieval (Miller and Matzel, 1988) or simple selective attentional process not coupled to error-correction learning (see Terao et al., 2015) has not been investigated. Slugs and snails possess well-developed central nervous systems (Sahley et al., 1981; Loy et al., 2006), comparable to those of insects, and it would be therefore likely that the blocking effect is based on error-correction learning rules as well. On the other hand, since the central nervous system of planarians is much less organized than that of insects, it would be likely that blocking in planarians reflects processes other than error-correction learning. In insects, it is of interest to see whether blocking is based on an error-correction rule in species other than crickets. However, unambiguous evidence of blocking phenomenon has not been found in honey bees (Guerrieri et al., 2005; Blaser et al., 2006, 2008) or in the fruit fly Drosophila melanogaster (Young et al., 2011). In the case of honey bees, for example, contradictory results have been reported in the literature from blocking of the CR (Smith, 1997; Hosler and Smith, 2000) to the absence of blocking (Blaser et al., 2006, 2008). Guerrieri et al. (2005) reported blocking, no blocking or even enhanced responding to the blocked element (i.e., augmentation) depending on the odor pairs used in the blocking experiment in honey bees. The reasons for the contradictory results in honey bees remain to be explored.

Finally, phenomena that are not consistent with the Rescorla–Wagner model, such as recovery from extinction, and phenomena that are difficult to be accounted for by the Rescorla– Wagner model without appropriate revisions, such as secondorder conditioning, have been reported in some invertebrate species (e.g., Sahley et al., 1981; Loy et al., 2006; Hussaini et al., 2007; Tabone and de Belle, 2011; Alvarez et al., 2014). What neural circuit mechanisms underlie associative learning in these species remains for future subjects.

#### AUTHOR CONTRIBUTIONS

MM, KT, and BA wrote the manuscript and approved the final version.

#### FUNDING

This study was supported by Grants-in-Aid for Scientific Research from the Ministry of Education, Science, Culture, Sports and Technology of Japan to MM (Nos. 16H04814 and 16K18586) and to KT (No. 15J01414) and by JSPS Postdoctoral Fellowship Program to BA (No. PE17047).

## REFERENCES

fpsyg-09-01272 July 19, 2018 Time: 16:32 # 7



**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2018 Mizunami, Terao and Alvarez. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Multiple Representations of Space by the Cockroach, Periplaneta americana

Matthew B. Pomaville and David D. Lent\*

Department of Biology, California State University, Fresno, CA, United States

When cockroaches are trained to a visual–olfactory cue pairing using the antennal projection response (APR), they can form different memories for the location of a visual cue. A series of experiments, each examining memory for the spatial location of a visual cue, were performed using restrained cockroaches. The first group of experiments involved training cockroaches to associate a visual cue (CS—green LED) with an odor cue (US) in the presence or absence of a second visual reference cue (white LED). These experiments revealed that cockroaches have at least two forms of spatial memory. First, it was found that during learning, the movements of the antennae in response to the odor influenced the cockroaches' memory. If they use only one antenna, cockroaches form a memory that results in an APR being elicited to the CS irrespective of its location in space. When using both antennae, the cockroaches resulting memory leads to an APR to the CS that is spatially confined to within 15◦ of the trained position. This memory represents an egocentric spatial representation. Second, the cockroaches simultaneously formed a memory for the angular spatial relationships between two visual cues when trained in the presence of a second visual reference cue. This training provided the cockroaches an allocentric representation or visual snapshot of the environment. If both egocentric and the visual snapshot were available to the cockroach to localize the learned cue, the visual snapshot determined the behavioral response in this assay. Finally, the split-brain assay was used to characterize the cockroach's ability to establish a memory for the angular relationship between two visual cues with half a brain. Split-brain cockroaches were trained to unilaterally associate a pair of visual cues (CS—green LED and reference—white LED) with an odor cue (US). Split-brain cockroaches learned the general arrangement of the visual cues (i.e., the green LED is right of the white LED), but not the precise angular relationship. These experiments provide new insight into spatial memory processes in the cockroach.

Keywords: vision, olfaction, allocentric memory, egocentric memory, visual snapshot, insect

# INTRODUCTION

The cockroach's environment is composed of a variety of sensory cues that convey important information about food, shelter, and danger. As the cockroach navigates through this sensory milieu it must be able to retain behaviorally relevant information. The utilization of internal and external cues facilitates the formation of proper associations about the relevant information,

#### Edited by:

Lars Chittka, Queen Mary University of London, United Kingdom

#### Reviewed by:

Claire Rusch, University of Washington, United States Stanley Heinze, Lund University, Sweden

> \*Correspondence: David D. Lent dlent@csufresno.edu

#### Specialty section:

This article was submitted to Comparative Psychology, a section of the journal Frontiers in Psychology

Received: 26 February 2018 Accepted: 09 July 2018 Published: 30 July 2018

#### Citation:

Pomaville MB and Lent DD (2018) Multiple Representations of Space by the Cockroach, Periplaneta americana. Front. Psychol. 9:1312. doi: 10.3389/fpsyg.2018.01312

**148**

thereby maximizing the cockroach's fitness. Integration of multimodal information and associative memory systems can function to signal spatially relevant information. This spatial information is assumed to be stored in the cockroach's brain and used to facilitate the localization of objects and places. It has been shown that the cockroach uses both olfactory and visual spatial information to localize relevant goals in its environment (Lent, 2006). The cockroach uses of the spatial structure of an odor stimulus for directional orientation (Hösl, 1990) and the ability to learn the spatial relationship between visual cues (Mizunami et al., 1998; Kwon et al., 2004).

The results from experiments examining associative and spatial learning in restrained cockroaches reveal that the manipulations made to the sensory conditions under which cockroaches are trained can influence the nature of the resulting memory (Kwon et al., 2004; Lent and Kwon, 2004; Pintér et al., 2005; Lent et al., 2007). The results of these experiments characterizing associative learning (Lent and Kwon, 2004; Pintér et al., 2005; Lent et al., 2007) and spatial learning (Kwon et al., 2004) suggest that the cockroach may be using an unidentified spatial frame of reference to localize behaviorally relevant information. Using the antennal projection response (APR) assay to study associative learning and memory (Lent and Kwon, 2004; Pintér et al., 2005) revealed that the duration of the memory depends on the sensory conditions under which cockroaches are trained. Lent and Kwon (2004) showed that following training using non-restricted sensory conditions, the memory for the association persisted for at least 72 h, thus indicative of long-term memory (Lent and Kwon, 2004). However, cockroaches that were trained to associate sensory information presented to the antenna and the eye on one side only (restricted sensory condition) demonstrated APRs that persisted for less than 24 h (Pintér et al., 2005), suggesting a failure to consolidate the association to longterm memory. However, when taken into consideration with the results of other experiments (Kwon et al., 2004; Lent et al., 2007), it may suggest that the way in which cockroaches were trained resulted in two different memories being established; one for the general association of the cues and one for the spatial location of the cue. Kwon et al. (2004) and Lent et al. (2007) revealed that the sensory conditions that cockroaches experience during learning affected memory for the position of the CS and support the hypothesis that different memory are being established. Kwon et al. (2004) looked at responses to the CS at positions other than the trained position. In this experiment, APRs were elicited only when the CS was within 15◦ of the learned position which is close to the angular sensitivity of the cockroach in darkadapted conditions (Heimonen et al., 2006) and it was suggested that the failure to show APRs toward these other positions may be due to the CS becoming ambiguous when moved in the environment. It has also been shown that following training with restricted sensory input, the APRs elicited from the side that did not receive odor or visual input during training were similar to the APRs elicited from the side that was trained and the memory was determined to be generalized (Lent et al., 2007). We hypothesize that these experiments are looking at two different types of memories. Cockroaches are either forming a memory for the simple association between two cues that is generalized

or forming a memory that is for the spatial location of the cue and the way in which the antennae interact with the environment is important in determining which of one of these memories is formed.

In addition to better understanding how the sensory conditions result in the establishment of spatial memories, the APR should be further explored in conjunction with the split-brain cockroach assay (Lent et al., 2007). By combining the split-brain assay with a modified version of the spatial learning assay (Kwon et al., 2004), we would have an assay that could be used in the future to characterize the role of central structures, such as the central complex, and lateral structures, such as the mushroom bodies in spatial learning and memory (e.g., Mizunami et al., 1998). The mushroom bodies have long been shown to be involved in learning and memory (Heisenberg, 2003), and several studies have shown them to be important for visual and olfactory spatial behaviors. The mushroom bodies have been linked to spatial behaviors in cockroaches (Mizunami et al., 1998), butterflies (Montgomery et al., 2016; van Dijk et al., 2017), ants (Stieb et al., 2010, 2012; Grob et al., 2017), foraging in honey bees (Farris et al., 2001), and thigmotaxis in Drosophila (Besson and Martin, 2005), but their role in spatial learning and memory could be better characterized. Other spatial memory processes are either bilaterally distributed or involve the central complex (Ofstad et al., 2011; Seelig and Jayaraman, 2013, 2015; Martin et al., 2015; Varga and Ritzmann, 2016; Dewar et al., 2017; Turner-Evans et al., 2017).

Discussed here are a series of new experiments using modifications of established paradigms to reveal different types of spatial memory in the cockroach. We aim to test four hypotheses: (1) The memory for the spatial location of a visual cue is made more precise when the cockroach is able to freely sample its environment with both antennae during the learning of a visual–olfactory association. (2) The movement of the antennae in response to the odor source is providing the spatial information that helps to establish a spatial frame of reference during the learning of a visual– olfactory association. (3) The cockroach can simultaneously store spatial memory representing the angular relationship between two visual cues and spatial memory for a single visual cue learned relative to the odor spatial frame of reference. (4) The cockroach can establish a memory for the angular relationship between two visual cues using only half a brain. From these experiments, two types of representation of spatial information are considered: (1) spatial cues that are represented in relation to the cockroach, and (2) spatial cues that are represented in relation to each other (Benhamou and Poucet, 1998). Here, we suggest that the cockroach uses the spatial information, derived from olfactory and motor/proprioceptive feedback from paired antennae movements, to learn cues with respect to its own body. This olfactory sampling/antennal movement-derived egocentric memory and, previously identified spatial memory for positional visual cues, i.e., allocentric or visual snapshot memory, can both be used to localize a cue in space. Additionally, we demonstrate that the cockroach can form a memory for the relative position of two visual cues with half a brain. The results

of these behavioral experiments provide a foundation to further explore the localization of spatial memory processes in the brain of the cockroach.

#### MATERIALS AND METHODS

## Animals

Experiments were conducted in Tucson, Arizona from 2005 to 2007 (Experiments 1 and 2) and in Fresno, California from 2012 to 2014 (Experiments 2 and 3) on adult male American cockroaches (Periplaneta americana) purchased from Carolina Biological Supply. The colony was maintained at approximately 25–28◦C on a 12:12 light–dark cycle at 50–60% humidity. Rearing cages were supplied with natural cat food (Taste of the Wild Pet Foods, Meta, MO, United States or IAMS, Dayton, OH, United States) and natural peanut butter (JIF Natural, The J.M. Smucker Company, Orrville, OH, United States or Skippy; Bestfoods, Co., Englewood Cliffs, NJ, United States). Individuals with damaged or missing appendages or antennae were rejected for testing.

After 48 h of food deprivation and isolation, animals were anesthetized using CO<sup>2</sup> and loaded into restraint tubes made from small polyethylene test tubes. The animals were secured with their heads and antennae exposed using a small dental wax collar, and the rear of the tube was sealed with laboratory parafilm. Animals secured in the restraint tubes were then placed into the testing room under red light and were left for at least 1 h to allow for recovery from the anesthetic. After the recovery period, the restrained animals were observed for natural antennae and leg movements. Animals displaying normal sampling (i.e., normal responses to air current, tactile stimulation, etc.) and a complete range of movement were moved into the training/testing arena for experimentation. In experiments using intact brain cockroaches, approximately 80% demonstrated normal antennal movement, and in the split-brain experiments, approximately 60% demonstrated normal antennal movements.

#### Split-Brain Cockroach

Animals to undergo split-brain lesioning procedure were prepared as described by Lent et al. (2007). Cockroaches were anesthetized with CO<sup>2</sup> and then restrained on a cold plate with their head immobilized using dental wax. An incision through the head capsule was made approximately 2 mm deep and 1– 1.5 mm in length using a small razor blade in a blade holder. The incision was sealed using a droplet of melted dental wax and the animals were allowed 48 h in isolation to recover. Following behavior experiments, split-brain cockroaches were dissected and the extent of lesion characterized. Only those cockroaches that had their brain completely split, with the exception of the subesophageal ganglia, were included in the analysis.

Non-lesioned control animals were anesthetized using CO<sup>2</sup> and were restrained on a cold plate in the same manner as animals undergoing the lesioning procedure. The control animals then had a drop of hot wax applied to the head in the same location as the lesion in non-control animals. Control animals were also placed in isolation cages for 48 h after the mock lesioning to prepare them for training and testing.

#### Arena

#### Experiments 1 and 2

As described in previous accounts (Kwon et al., 2004; Lent and Kwon, 2004), experiments were conducted in an arena enclosed within a visually uniform chamber illuminated with an infrared lamp. A restrained cockroach was positioned in the middle of the arena and aligned with respect to the green LEDs on the arena wall positioned at 15◦ intervals to the right and left of the insect (**Figure 1A**). The distance from the insect's head to the position of these cues was 15 cm. Each LED was given a number, 1–5. Five white LEDs (E1000, Gilway Technical Lamp, Co., Woburn, MA, United States) were positioned on the wall of the arena to the right and left of the insect. These contralateral reference stimuli (ConRS) were also spaced at 15◦ intervals with respect to the cockroach and named Z, A–D. Food odors controlled by a solenoid valve were presented through an odor delivery system positioned at green LED 1. Stimuli and their sequences were controlled by a Grass S88 stimulator (Grass Instrument, Co., Quincy, MA, United States). In all experiments, the US was presented for 1-s and the CS for 2-s using simultaneous conditioning. A ventilation system was placed above the arena to remove odor after each trial (see Lent and Kwon, 2004 for details).

#### Experiment 3

The arena used was based on the design used by Kwon et al. (2004) and Lent and Kwon (2004) with some modifications to allow for multiple testing angles to be explored. The arena was formed using a 30 cm diameter wooden ring with vertical pairs of green (520–525 nm, 20,000 mcd, and C-LEDs) and white (6,000K, 20,000 mcd, and C-LEDs) LEDs every 15◦ from the centerline to 75◦ off center (**Figure 1A**). At the 75◦ position, a small polyethylene tube attaches to a syringe filled with an odor source (JIF Peanut Butter). Pure air puffs (charcoal filtered; air pressure 1 atm; and stimulus duration 1 s) were blown through the syringe cartridge containing the odor using a solenoidcontrolled air source. All timing of lights and odor is done with a pair of Velleman MK188 Pulse-Pause timers (Velleman NV, Legen Heirweg 33, B-9890 GAVERE, Belgium, Europe). In all experiments, the US was presented for 1-s and the CS for 2-s using simultaneous conditioning. The odor concentration being delivered was only measured by observing behavioral responses. Permanent air flow was provided by an exhaust fan system placed above and behind the arena to remove odors from the inside of the arena between trials, and the surface of the arena cleaned with ethanol.

#### Experiment 1: Training in Non-restricted, Restricted, and Semi-Restricted Sensory Conditions

Using the protocol and statistical analysis described by Lent and Kwon (2004) and Lent et al. (2007), intact brain cockroaches were conditioned with either non-restricted, restricted, or semirestricted sensory input (**Figure 1A**). In all conditions, the

protocol consisted of two pretraining trials of a two-second presentation of the green LED to both the left and right side of the animal at position 1 for a total of four pretraining trials

(see **Figure 1B**) with a 1-min interval. Pretraining measured the cockroaches' baseline response to the conditioned stimuli (CS). Animals which showed APR to the CS in all trials were rejected

stimulus (IpsiRS).

from further trials. Fewer than 10% of cockroaches elicited APRs to the CS in all pretraining trials.

After pretraining, the cockroaches were randomly trained to either position 1 on the right or position 1 on the left. Training comprised five trials of the green LED (CS) paired with the food odor, the unconditioned stimulus (US), as described by Lent and Kwon (2004). After the training was completed, animals responding to three or more presentations (60–72%) were isolated under a black cup for 15 min before testing to allow for the memory to be represented in a way that the APR could be elicited on the side opposite of the trained side (Lent et al., 2007). After 15 min, cockroaches were tested for the presentation of the CS at positions 1–4 on both the trained side and the opposite side in a random order. The CS was presented for 2 s, and APRs were measured for 30 s. The time interval between tests was 1 min.

If cockroaches were conditioned with non-restricted sensory input, both antennae could freely move and sample the olfactory environment. Additionally, they did not have any visual obstruction to the eye opposite of training (**Figure 1A**). If cockroaches were conditioned with restricted sensory input, the antenna on the opposite side of that given the CS + US pairing was secured with wax at the base and covered with a capped polyethylene tube. Additionally, the eye on the opposite side was covered with opaque wax (**Figure 1A**). Semi-restricted sensory conditioning involved three assays which either blocked visual input, antennal movement, or olfactory input on the opposite side of that given the CS + US pairing (**Figure 1A**). The semirestricted sensory input assays were designed to examine the role of different sensory modalities. The first assay involved restricting only visual input to one eye while permitting antennal input. The second assay restricted proprioceptive reafferent sensory input by fixing the base of the antenna with wax and thus restricting movement of one antenna while allowing olfactory input. The third restricted olfactory input by covering the antenna with a thin film of light mineral oil (Fisher Scientific) allowing the animal to move its antenna, while reducing (<30% APR) significantly the ability of that antenna to sample odor (oil vs. normal response to odor: n = 15, z = 1.55, and P = 0.0128).

#### Experiment 2: Training in the Presence of a Contralateral Reference Stimulus

Cockroaches were trained to associate an odor cue (US) with a green LED (CS) in the presence of a white LED reference stimulus on the contralateral side (ConRS) using the protocol described by Kwon et al. (2004) (**Figure 1B**). During training, the ConRS was on throughout the trial, unless otherwise noted. Training comprised two pretraining trials with the ConRS at position C, and the CS at position 1 to replicate the procedure described by Kwon et al. (2004). This was followed by five training trials of the ConRS and CS + US at positions C and 1, respectively. After the training was completed, animals responding to three or more presentations (65%) were isolated under a black cup for 15 min before testing. The testing phase comprised eight presentations of the CS and ConRS: two trials presented the ConRS and CS where the angular relationship was the same as training (C and 1; A and 3), two trials presented the ConRS and CS with angular relationships different from training (Z and 2; A and 4), and four trials of the CS alone in the absence of the ConRS at positions 1, 2, 3, and 4 (see **Figure 1**). The CS was presented for 2 s and APRs were measured for 30 s. Between each trial, the cockroach was covered with a black cup so that the ConRS position could be changed. The time interval between tests was 1 min. With the exception of the first test, which was always at the trained position, test position order was randomized.

# Experiment 3: Training in the Presence of an Ipsilateral Reference Stimulus (IpsiRS)

Intact brain and split-brain cockroaches were trained to associate an odor cue (US) with a green LED (CS) in the presence of a white LED reference stimulus displaced 30◦ medially on the ipsilateral side (IpsiRS) (**Figure 1B**). During training, the IpsiRS was on throughout the trial, unless otherwise noted. The protocol consisted of four pretraining trials with the IpsiRS at position B and the CS at position 1, presented two times to each side. Following pretraining, cockroaches were randomly assigned to the non-restricted or restricted group. The restricted group had the eye and antenna on one side, side randomly selected, covered. Cockroaches were given five training trials pairing the IspiRS and CS + US at positions B and 1, respectively. After the training was completed, animals responding to three or more presentations (60% intact brain and 46% split-brain) were isolated under a black cup for 15 min before testing.

Testing comprised 10 total trials. Nine trials were presentations to the trained half and one was to the naïve half. Tests to the trained half included: (1) three trials testing the APR when angular relationship between the IpsiRS and CS was the same as training, (2) two trials when the angular relationship between the IpsiRS and CS was greater than training, (3) two trials when the angular relationship IpsiRS and CS was smaller than training, and (4) one trial when the positions of the lights were swapped. Tests to the naïve half included: one trial testing the APR when the angular relationship is maintained, but mirrored, to test if the memory was generalized in a way that the cockroach remembered the IpsiRS was located anteriorly to CS. Due to the length of the testing period, all tests concluded with a trial testing the APR toward the original trained position on the trained half. This was to ensure that any lack of response was not due to fatigue or extinction of the learned response. Because of the time required to change the positions of the visual cues, 2-min intervals between tests were used. With the exception of the first and last test, which was always at the trained position, test position order was randomized. Finally, an experiment was done with intact brain cockroaches that were trained in the non-restricted sensory condition and tested at the trained position and angle but rotated to the contralateral side. In this rotated test to the contralateral side, the white LED (IpsiRS) was at position A and the green LED (CS) was at position 4 (**Figure 1A**). This was done to test if the response to the angular relationship was maintained even when rotated to the opposite side.

## Data Collection and Statistics

Data were collected through direct observation of the APR in restrained cockroaches as viewed through a video feed. To aid in accuracy, the floor of the arena had marks at each at ±3 ◦ CS position to give a window to score the APRs similar to that defined by Kwon et al. (2004). APRs were measured and analyzed as described by Kwon et al. (2004), Lent and Kwon (2004), and Lent et al. (2007). In each trial, cockroaches were given 30 s to respond to the stimulus. Only if the first movement of APR was directed toward the odor source location or to the CS (±3 ◦ ) was it scored as a "1." If the cockroach's APR was toward a position other than the CS, if the cockroach struggled in the restraint during the stimulus, or if the antenna did not move from baseline in response to the stimulus, the response was scored as a "0." The results from experiments were analyzed using non-parametric statistics. The Freidman's test, Wilcoxon Signed-Rank test, or Wilcoxon Rank Sum test was used to identify significant difference from the pretrained response rate and differences between tests. The F-test of equality of variance was used to analyze the timing of contralateral antennal recruitment in intact and split-brain cockroaches. All statistical tests were run using MATLAB 2017A (Mathworks, Inc.).

### RESULTS

### Visual–Olfactory Associations Reveal an Underlying Spatial Component

The first hypothesis tested was that the memory for the spatial location of the visual cue is made more precise when the cockroach is able to freely sample its environment with both antennae during the learning of a visual–olfactory association. First, we examined if the association was generalized to the contralateral side of cockroaches that were conditioned with non-restricted sensory input and CS + US at position 1. The APRs were measured from the same side as training (trained half) at position 1 and the opposite side of training (naïve half) at position 1. APRs elicited from the "naïve half " of cockroaches were not statistically different from those elicited from untrained cockroaches (n = 18, Signed-Rank, P = 0.5) and, thus, the memory was not generalized. To examine the hypothesis of precision due to the presence of spatial information versus ambiguity due to movement of the CS in the environment, additional experiments were performed. Cockroaches were trained with either non-restricted or restricted sensory input to associate a visual cue and an olfactory cue. The training cues were offset 75◦ right from the midline. For testing, the cues were at positions 75, 60, 45, and 30◦ (positions 1, 2, 3, and 4, respectively), both right and left of midline. In the non-restricted sensory condition, APRs elicited following training were significantly different when the CS was presented at positions 1 and 2, but not at the other positions or on the contralateral side (**Figure 2A**). In the restricted sensory condition, APRs elicited following training were significantly different from pretraining. However, the APRs elicited from each individual position were not significantly different from each other (**Figure 2B**).

# Antennae Sampling Behavior Provides a Spatial Frame of Reference

Next, we tested the hypothesis that the movement of the antennae in response to the odor source delivered during conditioning is providing spatial information, establishing a spatial frame of reference during training. To address the underlying role of sensory processing in providing spatial information resulting in an APR that is spatially localized, the cockroaches were conditioned using paradigms that provide varying degrees of sensory restriction. This is designated here as conditioning with semi-restricted sensory input. Cockroaches trained under the first semi-restricted sensory condition (vision) elicited APRs to visual cues at different positions in a similar fashion to cockroaches trained under non-restricted sensory conditions. APRs were elicited only when the visual cue was tested at positions 1 and 2 and did not elicit APRs to a visual cue presented on the side opposite of that trained (**Figure 2C**). As visual input is already restricted by the design of the paradigm to one hemifield [outside of the binocular region (Seelinger and Tobin, 1981)], this result was not unexpected. It also demonstrated that the presence of the eye shield itself and any mechanical feedback that it may convey was not interfering with learning and memory processes in this assay. Cockroaches trained under both the second (movement) and third (olfactory) semi-restricted sensory conditions demonstrated APRs that were similar to cockroaches trained under restricted sensory conditions. When only the movement of the antenna was blocked, the cockroaches elicited APRs toward visual cues irrespective of where the cue was positioned (**Figure 2D**). APRs toward visual cues were significantly different from pretraining at all positions and APRs elicited at different positions were not significantly different from each other. Similarly, when only olfactory information was blocked cockroaches elicited strong APRs toward visual cues positioned in either hemifield (**Figure 2E**). These APRs were significantly different from pretraining, but not significantly different from each other.

The results of varying the degrees of sensory restriction during learning suggest recruitment of the contralateral antenna is important. To better understand how the contralateral antenna may be contributing to sampling the odor cue, we use the split-brain assay which has been shown to decouple antennal movements (Lent et al., 2007). Given that cockroaches only show spatially restricted APR to a single cue when both antennae are able to freely move and sample the odor, we hypothesized that non-restricted sensory conditioning results in the quicker recruitment of the contralateral antenna and the coupling of antennal movements that may provide the idiothetic cues necessary for the establishment of an egocentric spatial frame of reference. Here, we looked at the time to the recruitment of the contralateral antenna from the onset of odor and compared the response in split-brain (n = 24) and intact brain (n = 30) cockroaches. When recording the horizontal position of the tips of the antennae of restrained cockroaches at rest, the movements of intact brains are typically synchronous whereas those of

APRs were not significantly different (z = –0.327, P = 0.7437). The APRs were significant when presented at the different trained positions (T1–T4) (χ P = 5.46E−<sup>6</sup> no differences in the response to T1 vs. T2 (z = 0.6498, P = 0.5158). APRs were not different from pretraining when tested at other positions (T3 and T4 – χ <sup>2</sup> = 6, P = 0.116; O1–O4 – χ <sup>2</sup> = 4, P = 0.2615). (B) APRs of cockroaches (n = 24) trained in the restricted sensory condition. The pretraining APRs were not significantly different (z = –0.3883, P = 0.6978). The APRs toward the CS were significantly different from pretraining (χ <sup>2</sup> = 79.64, P = 1.9E−13), but were not different from each other at any of the positions (T1–T4 – χ <sup>2</sup> = 12.35, P = 0.0895 and O1–O4 – χ <sup>2</sup> = 8.68, P = 0.1223). (C) The APRs of cockroaches (n = 24) trained in the vision semi-restricted sensory condition. The pretraining APRs were not significantly different (z = –0.351, P = 0.7258). The APRs were significantly different between the tests (χ <sup>2</sup> = 68.44, P = 3.045E−12). The APRs to the CS that was within 15◦ of the learned position (T1 and T2) were different from pretraining (χ <sup>2</sup> = 135.95, P = 7.7E−<sup>8</sup> ), but not each other (z = 0.6154, P = 0.5383). APRs were not significant at other locations (T3 and T4 – χ <sup>2</sup> = 3.0, P = 0.2232; O1–O4 – χ <sup>2</sup> = 7.3333, P = 0.1193). (D) APRs of cockroaches (n = 24) trained in the antenna movement semi-restricted sensory condition. The pretraining APRs were not significantly different (z = –0.7505, P = 0.4529). The APRs elicited to the CS positions were significantly different from pretraining (χ <sup>2</sup> = 91.5, P = 8.14E−16), but were not different from each other (T1–T4 and O1–O4; χ <sup>2</sup> = 11.2, P = 0.1301). (E) APRs of cockroaches (n = 24) trained in the olfaction semi-restricted sensory condition. The pretraining APRs were not significantly different from each other (z = –0.351, P = 0.7258). The APRs elicited to the CS positions were significantly different from pretraining (χ <sup>2</sup> = 73.59, P = 3.00E−12), but were not different from each other (T1–T4 and O1–O4; χ <sup>2</sup> = 13.18, P = 0.0679). Bar colors and letters reflect statistical groups. Illustration above graphs represents the position of cues during experiment.

the split-brain are asynchronous (**Figure 3A**, top). To test the recruitment of the contralateral antenna to an odor stimulus, cockroaches were presented with a single 2-s pulse of odor at the 45◦ position, and the time it took each antenna to begin sampling was measured (**Figure 3A**, bottom). The recruitment of the contralateral antenna was significantly delayed in the splitbrain cockroaches compared to the intact brain cockroaches (**Figure 3B**).

# Parallel Memory Processes and Spatial Representations

With evidence to suggest cockroaches can localize a single cue in space when allowed to freely sample the environment, we tested the hypothesis that cockroaches can simultaneously store a spatial memory representing the angular relationship between two visual cues and a spatial memory for a single visual cue learned relative to the egocentric reference. Cockroaches learn to associate the CS + US in the presence of a contralateral reference visual cue (Kwon et al., 2004) but, do they also learn the CS using the egocentric frame of reference? To address this question, cockroaches were trained to associate a visual cue with an olfactory cue in the presence of a ConRS. The APRs of cockroaches were then tested both in the presence and absence of the reference cue. Cockroaches learned the spatial relationship between the ConRS and the CS similarly to those previously described (**Figure 4**). When the CS was presented at varying positions in the absence of the reference, however, they elicited APRs only in a limited region of space (**Figure 4**). Thus, cockroaches elicited APRs only if the CS was presented within 15◦ of the trained position. However, cockroaches would respond to the CS outside of this 15◦ range when the CS was coupled with the ConRS.

These findings lead to another question: can cockroaches learn just the angular relationship between two visual cues and not the egocentric-derived spatial representation? To address this question, cockroaches were conditioned with semi-restricted sensory input that blocked movement of, and olfactory input to the antenna on one side by covering it with a small tube, while permitting visual input to both eyes. These cockroaches were trained to associate the visual cue with the olfactory cue in the presence of a ConRS. Cockroaches trained under these conditions did not demonstrate either form of spatial learning; their APRs were similar to those classically conditioned with restricted sensory input. The APRs of cockroaches trained in this condition were significantly higher than pretraining (n = 24, χ <sup>2</sup> = 16.4444, and P = 0.00248) and not significantly different from each other (n = 24, χ <sup>2</sup> = 4.50, and P = 0.2123). Thus, cockroaches elicited APRs irrespective of where the CS was presented during the different tests.

### Spatial Learning Localized to Half of the Brain

Given that we could not separate the two forms of spatial memory in the intact brain cockroach, we wanted to examine the limits of the cockroaches' abilities to establish a memory for the angular relationship between two visual cues. To test this limit, we characterized spatial memory in the split-brain cockroach. We hypothesized that cockroaches can establish a memory for the angular relationship between two visual cues using only half a brain. Split-brain cockroaches (N = 39) that were trained with non-restricted sensory conditioning showed significant APRs toward the CS paired with the IpsiRS when the angular relationship closely matched that of learning (**Figure 5A**). When the angular mismatch between the CS and IpsiRS was too large or too small, the APR was significantly lower than the trained angle response but was also significantly different from the pretraining response. If the position of the CS and the IpsiRS were swapped, APRs were significantly reduced and were similar to those observed in pretraining. The cockroaches were given a single presentation of the mirrored cue combination to the side

FIGURE 4 | Antennal projection responses demonstrate multiple representations of space. The APRs of cockroaches (n = 24) that were trained in the presence of a contralateral reference cue were examined with different configurations of the ConRS and CS. When tested, the APR to the CS in the presence of a ConRS were significantly different (χ <sup>2</sup> = 22.67, P = 4.739E−<sup>5</sup> ). When tested at positions that maintained the ConRS–CS training relationship, APRs were significantly different from pretraining (χ <sup>2</sup> = 18.73, P = 8.58E−<sup>5</sup> ), but not different from each other (z = 0.5777, P = 0.5643). When tested with other configurations that had larger or smaller angular relationships the APRs were not significantly different from each other (z = 0.6154, P = 0.5383) or pretraining (χ <sup>2</sup> = 4.67, P = 0.097). When tested without the ConRS and only the CS at positions 1–4, the APRs were significantly different from each other (χ <sup>2</sup> = 20, P = 0.0002). There were significant APRs when the CS was presented within 15◦ of the learned position (1 and 2), but not when the CS was presented at other locations (3 and 4) (1 and 2 vs. pretraining, n = 24, χ <sup>2</sup> = 16.23, P = 0.0003; 3 and 4 vs. pretraining, n = 24, χ <sup>2</sup> = 3, P = 0.2231). The APRs to positions 1 and 2 were not significantly different from each other (z = 0.2759, P = 0.7826), nor were the APRs to positions 3 and 4 (z = –0.308, P = 0.7581). Bar colors and letters reflect statistical groups. The illustrations above the graphs represent the position of cues during the experiment.

opposite training to check for possible memory generalization (i.e., white light is anterior to green light) as demonstrated by Lent et al. (2007) and cockroaches demonstrated a significantly lower APR toward the cue compared to pretraining. Finally, cockroaches were given a test at the original training position and they demonstrated a strong APR, suggesting that the memory was still intact and the prolonged testing procedure did not result in diminishing the response due to lack of reinforcement (**Figure 5A**). As a comparison, cockroaches that had an intact brain, but underwent mock surgery were trained. When intact brain cockroaches (N = 25) were trained using non-restricted sensory conditions, the responses to the trained angular relationship of intact brain cockroaches were similar to those observed in the split-brain cockroach. The cockroaches showed significant APRs when presented with the CS in the presence of the IpsiRS (**Figure 5B**). Again, similar to the splitbrain cockroaches, the intact brain cockroaches elicited an APR to the CS paired with the IpsiRS at both larger and smaller angles. The percentage of APRs toward the larger and smaller CS and IpsiRS angular relationships was the less than as learned relationship, but greater than pretraining. If the position of the CS and IpsiRS were swapped, the APRs were reduced to pretraining levels, as they were when the CS and IpsiRS were presented mirrored to the side opposite of training (**Figure 5B**). Finally, we trained a group of intact brain cockroaches (N = 11) using non-restricted sensory condition to associate the CS + US in the presence of the IpsiRS and performed two tests. One test was with the CS and IpsiRS at the same position as training and one test with the same angular relationship as training but rotated to the contralateral side. Cockroaches showed significant APRs only at the trained position and not when the paired visual cues were rotated to the contralateral side (**Figure 5C**).

Given that cockroaches that are trained to associate the CS + US in the presence of a ConRS fail to establish a memory for the angular relationship when antenna is restricted, we wanted to test if the same was true for intact brain cockroaches trained to the CS + US in the presence of an IpsiRS. Intact brain cockroaches that were trained using restricted sensory conditions (N = 30) demonstrated APRs similar to the APRs of intact brain cockroaches trained using non-restricted sensory conditions. The APRs were significantly different from pretraining when the angular relationship was the same, larger, and smaller (**Figure 6**). Similarly, the APR toward larger and smaller CS and IpsiRS angular relationship was significantly different from pretraining. Only when the position of the CS and IpsiRS was swapped or mirrored on the opposite side did APRs reduce back to pretraining levels. Contrary to what is observed when using restricted sensory conditioning with no reference cues, the memory of the CS-IpsiRS was not generalized to the other side and there was not a significant APR when the CS and IpsiRS were presented on the opposite side of that which was trained (**Figure 6**).

# DISCUSSION

# Odor Spatial Frame of Reference

These results, combined with previous accounts (Kwon et al., 2004; Lent and Kwon, 2004; Lent et al., 2007), suggest that cockroaches trained with a restricted sensory input elicit APRs to the visual cue regardless of its position, whereas cockroaches trained with non-restricted sensory input elicit APRs only in the trained hemifield and only if the visual cue does not deviate drastically from the learned position. From the new experiments described in this paper and those described earlier (Kwon et al., 2004; Pintér et al., 2005; Lent et al., 2007), two hypotheses regarding the memory of the learned visual–olfactory association can be developed. First, a single antenna processing olfactory stimuli is sufficient to associate an olfactory cue with a spatially coincident visual cue. In this assay, this association is generalized and the visual cue is indicative of an odor irrespective of where

significant (χ <sup>2</sup> = 158.8, P = 1.321E−29). The first and last test APRs were different from pretraining (first test: z = –6.14, P = 8.23E−10, last test: z = –5.86; P = 4.58E−<sup>9</sup> ), but not each other (z = 0.2655, P = 0.7906). APRs in tests that maintained the angular relationship were similar to each other (z = 1.304, P = 0.1923), different from pretraining (z = –7.18; P = 7.05E−10) and similar to the first and last test (P > 0.30). When tested at position with larger angular relationships, APRs were similar to each other (z = 0.2204, P = 0.8254), as was the APRs in tests at the smaller positions (z = –0.2195, P = 0.8262). The APRs in larger and smaller tests were not different from each other (z = –0.956, P = 0.03391), but were lower than the trained angle response (same vs. large: z = 2.68, P = 0.007; same vs. small z = –3.61, P = 3.1E−<sup>4</sup> ) and greater than pretraining (large: z = –4.32, P = 1.52E−<sup>5</sup> ; small: z = –3.22, P = 0.0012). APRs in the swapped and mirrored position tests were similar to or decreased from pretraining (Swap: z = 1.76, P = 0.078; Opp. Mirror: z = 2.48, P = 0.013). (B) The APRs of intact brain cockroaches (n = 25) during pretraining were not significantly different (z = 0.239, P = 0.811). The APRs elicited during testing were significant (χ <sup>2</sup> = 69.8, P = 1.669E−11). The first and last test APRs were different from pretraining (first test: z = –5.25, P = 1.54E−<sup>7</sup> , last test: z = –4.19; P = 2.77E−<sup>5</sup> ), but not different from each other (z = 0.9043, P = 0.3658). APRs in tests that maintained the angular relationship were similar to each other (z = 0.0, P = 1) and the first and last tests (P > 0.23), but greater than pretraining (z = –4.02, P = 5.73E−<sup>5</sup> ). The APRs to larger angles were similar to each other (z = 0.2697; P = 0.7874) as were the APRs to the smaller angles (z = 0.2723, P = 0.7854). The APRs to the larger and smaller angular relationship were similar to each other (z = –0.397, P = 0.691) and greater than pretraining (Lrg. Ang.: z = –3.16, P = 0.0016; Sm. Ang.: z = –2.69, P = 0.0072), similar to the same angle tests (Eq. Ang. vs. Lrg. Ang.: z = 1.27, P = 0.204; Eq. Ang vs. Sm. Ang.: z = 1.6, P = 0.11) and decreased compared to the first trained position test (Lrg. Ang.: z = 2.443, P = 0.0145; Sm. Ang.: z = 2.757, P = 0.0058), but not the last test (Lrg. Ang.: z = 1.454, P = 0.458; Sm. Ang.: z = 1.778, P = 0.0754). APRs in the swapped and mirrored position tests were similar to pretraining (Swap: z = 0.9161, P = 0.105; Opp. Mirror: z = 0.9161, P = 0.105). (C) The APRs of intact brain cockroaches trained with non-restricted sensory input were not significantly different from each other during pretraining (z = –0.3527, P = 0.7243) and during testing were only different from pretraining to the first test using the trained positions of the CS + IpsiRS (1 + B: z = –3.831, P = 1.27E−<sup>4</sup> ). When tested with the same angular relationship but rotated to the contralateral side, APRs were not significantly different from the pretraining (z = –0.47, P = 0.638) and significantly different from the other test position (z = 2.47, P = 0.0134). Bar colors and letters reflect statistical groups. Illustration above graphs represents the position of cues during experiment.

it appears in the environment. Second, both antennae sampling information from an odor source results in providing not only directional information but also positional information. The olfactory cue's positional information is detected with respect to the cockroach itself, presumably because it bilaterally processes and integrates olfactory and motor/proprioceptive information.

FIGURE 6 | Intact brain cockroaches trained in the restricted sensory condition unilateral spatial learning assay. The APRs of intact brain cockroaches (n = 30) during pretraining were not significantly different (z = 1.014, P = 0.310). The APRs elicited during testing were significant (χ <sup>2</sup> = 112.68, P = 4.18E−20). The first and last test APRs were significantly different from pretraining (first test: z = –7.07, P = 1.60E−12, last test: z = –5.47; P = 4.46E−<sup>8</sup> ), but not each other (z = 1.411, P = 0.1582). APRs in tests that maintained the angular relationship were significantly different from pretraining (z = –7.74, P = 9.90E−15), but not each other (z = 0.5693, P = 0.5691) or from the trained positions tests (P > 0.26). The APRs to larger angles were similar to each other (z = –0.2545; P = 0.7991) as were the APRs to the smaller angles (z = –0.251, P = 0.8018). The APRs to larger angles were similar to each other (z = 0.368, P = 0.713), were greater than pretraining (Lrg. Ang. – z = –6.39, P = 1.69E−10; Sm. Ang. – z = –5.99, P = 2.04E−<sup>9</sup> ), and similar to the same angle tests (Eq. Ang. vs. Lrg. Ang.: z = 1.355, P = 0.175; Eq. Ang vs. Sm. Ang.: z = 1.721, P = 0.085). The larger angle tests were similar to both the first (z = 1.739, P = 0.082) and last (z = 0.1478, P = 0.802) trained position test. APRs in the smaller angle tests were decreased compared to the first trained position test (z = 2.022, P = 0.0431), but not the last (z = 0.4488, P = 0.6535). APRs in the swapped and mirrored position tests were similar to pretraining (Swap: z = –1.089, P = 0.277; Opp. Mirror: z = –0.699, P = 0.4841). Bar colors and letters reflect statistical groups. Illustration above graphs represents the position of cues during experiment.

Varga and Ritzmann (2016) demonstrated that the cockroach, Blaberus discoidalis, encodes head direction using idiothetic cues in the absence of external cues. In our current work, the bilateral movement of the antenna during olfactory sampling may be providing the necessary idiothetic cues to establish the egocentric frame of reference. We hypothesize that the movement of the antennae results in the creation of an idiothetic frame of reference that can be used to learn the position of the CS relative to the cockroach. The possibility that there were any visual cues other than the green LED in the environment conveying spatial information, thereby creating additional visual landmark references, can be ruled out as all experiments were performed under infrared light conditions (a non-visible wavelength for cockroaches), and the training arena and surrounding area were visually uniform. Even though cockroaches are restrained and the retinotopic array of the eyes should be sufficient to provide all the spatial information needed, cockroaches only respond to the CS within a limited range if both antennae are able to move freely. We suggest that additional spatial information is being provided by the olfactory cue and the movement of both antennae in response to the odor which may help to reinforce the spatial information provided by the retinotopic organization of the input to the eyes. However, this needs to be further examined.

When analyzing the movements of antennae, the recruitment of the contralateral antenna provides additional insight into the behavioral response in the restrained assay. The baseline movements of the antenna are synchronous in the intact brain and asynchronous in split-brain restrained cockroaches in the absence of any delivered chemosensory or mechanosensory stimuli. Both synchronous and asynchronous movements in cockroaches are common. Cockroaches typically show stronger spatio-temporal coupling during walking rather than pausing (Okada and Toh, 2004). The increased spatio-temporal coupling observed in our assay (restrained = pausing) may be resultant of the design of the experiments, where cockroaches are restrained and not walking. When an odor stimulus was delivered, there were differences in the responses of the intact brain and splitbrain cockroaches, with the contralateral side being recruited faster in intact brain cockroaches. The observations of antennal movements in the restrained condition suggest that coupling of antenna movements require bilateral and/or centralized control processes. In the split-brain cockroach, these control processes may be disrupted, and thus may affect recruitment of the contralateral antenna in response to an odor presentation. This early recruitment may be important in providing a spatial frame of reference and deserves further consideration.

### Multiple Spatial Memories

Cockroaches elicit an APR that are spatially constrained when they are conditioned with both antennae free to sample the olfactory environment. The cockroaches, simultaneously, represented space in terms of the angular relationships between visual cues. This visual snapshot memory for the angular relationship between the two cues provides a memory that allows the cockroach to elicit an APR when presented with a similar angular arrangement of the visual cues during tests. By using snapshot or image matching the cockroach can compare its current view with the memory for the angular relationship of the two cues and only elicit an APR if the overall image similarity is high (Zeil et al., 2003). Thus, the snapshot memory representing the angular relationship of two visual cues further contributes to the cockroaches' ability to localize learned cues and has been proposed as a mechanism to facilitate visual navigation in insects (Collett and Collett, 2002). When both representations of space can be utilized by the cockroach to localize a learned cue, the memory for the angular relationship between the CS and reference stimulus must be the one that determines the behavioral response. This response may be unique to this particular behavioral assay. When olfactory sampling was blocked using semi-restricted sensory conditioning on the contralateral side of intact brain cockroaches, and they were trained to associate the angular relationship between the two cues (CS + ConRS), the cockroaches failed to learn the angular relationship. The cockroaches' APRs following training in this

condition were toward CS + ConRS angular relationships that were the same, smaller, and larger, as well as, to the CS alone at all positions. Interestingly, the cockroaches' response to the CS + ConRS is similar to what we see in cockroaches conditioned to the IpsiRS + CS, which also elicit APRs to angular relationships that are the same, smaller, and larger than the trained angular relationship. The failure of the cockroach to learn the precise angular relationship of the CS + ConRS in this semi-restricted condition could be due to one of two things. First, the learning is sequential and formation of the memory for the angular relationship of the CS + ConRS requires the other spatial frame of reference to be established first, which cannot be done when the antenna is restricted. Second, the design of the experiment intricately links olfaction and vision in the training paradigm, such that the formation of both types of spatial memory requires bilateral olfactory processing. We believe that the restriction of the antenna is the constraining factor and bilateral processing results in increased precision when learning the location of the visual cues. While the two types of spatial memory can be experimentally separated, as demonstrated in the experiments described above, it is unlikely that the learning of the two spatial representations can be experimentally separated using this paradigm.

### Spatial Learning and Memory in Half a Brain

The APR of the split-brain cockroaches demonstrated the acquisition of a learned unilateral spatial association. The results of this testing show that in split-brain models, the angular association between the ipsilateral reference light and the conditioned light can elicit an APR, even if the position of the lights is changed relative to the position of the cockroach. Previous studies show that the split-brain cockroaches perform as well as intact brain cockroaches during conditioning of the APR (Lent et al., 2007). It is known that there should not be any rotation of the CS more than 15◦ from its original position because the APR will be diminished (Kwon et al., 2004). However, this performance is improved by coupling the CS with a reference cue (Kwon et al., 2004). A similar improvement is seen in the split-brain cockroach and most clearly demonstrated when the position of the light cues was swapped. Another key finding is that when intact brain cockroaches are unilaterally trained using a spatial learning protocol, if the paired light cues are rotated into the contralateral field the cockroaches no longer elicit APRs. Additionally, the generalization of the memory, as tested by a mirroring of the cues, from the trained side to the untrained side is blocked and while the response in this test is even less than pretraining, it is significantly lower compared to what is expected from a positive response, reflecting the expected non-response during presentation of the cues. Perhaps, the additional cues provided help by giving an additional frame of reference to the cockroaches, thus allowing them to learn on which side the odor should be expected when they encounter the light cues in their environment. The ability of the brain to learn unilaterally may be a general phenomenon, because it has been shown that, in honey bees, the two brain halves can learn quite different association tasks independently, if each side is shielded from the stimuli presented to the other side (Sandoz and Menzel, 2001). It was expected that the immobilized animal could achieve monocular spatial memory. However, we did not know if this would require the integrity of both brain halves or whether this could be achieved after midline sectioning. Earlier studies (Mizunami et al., 1998) showed that place memory is abolished only when both mushroom bodies are lesioned and can be achieved as long as one mushroom body on the same side is undamaged. Ofstad et al. (2011) demonstrated that the central complex is necessary for spatial learning and place memory and an increasing number of studies have shown that orientation, visualguided behaviors, and landmark recognition depend on the central complex (Seelig and Jayaraman, 2015; Dewar et al., 2017; Stone et al., 2017; Turner-Evans et al., 2017). When all of our experiments are taken as a whole, the results provide support for the role of both central complex and mushroom bodies in spatial memory.

# Spatial Memory in the Cockroach

The APR assay can be used with varying degrees of sensory restriction in the intact brain cockroach or the split-brain cockroach and provides us with a number of behavioral protocols to examine associative and spatial learning and memory processes. Given the APR assay was designed to be used in a restrained cockroach, it provides a platform for electrophysiological studies which will allow us to better understand the neural basis of these behaviors. The organization of spatial memory and the dynamics of memory transfer still needs additional investigation in order to better understand how such processes are organized in the brain of insects. Importantly, the results of these and previously published (Kwon et al., 2004; Lent and Kwon, 2004; Pintér et al., 2005; Lent et al., 2007) experiments examining the APR in the restrained cockroaches suggest that these processes are distributed both unilaterally and bilaterally/centrally (**Figure 7**) which may provide us with brain areas to target.

Some experiments have been done to localize learning and memory processes to specific brain areas in P. americana. Associative memory processes are likely localized in the mushroom bodies and can be generalized from one side to the other over time (Lent et al., 2007). Mizunami et al. (1998) provided evidence for a neural basis of spatial learning in the cockroach. Cockroaches are able to use visual cues to learn the location of a hidden cool spot on a heated floor, but they have significantly reduced spatial learning and memory when the mushroom bodies have been lesioned bilaterally (Mizunami et al., 1998). This work suggests that spatial learning takes place either through communication between or convergence on the same output center of the paired mushroom bodies found in each brain hemisphere. A number of other studies have also provided evidence that the mushroom bodies may be important for spatial and visual behaviors. It has been shown that there is correlation between the plasticity of the mushroom body calyces and the size and spatial complexity of host range in the butterfly, Polygonia c-album (van Dijk et al., 2017) and spatial orientation is related to calyx expansion in Helioconius

butterflies (Montgomery et al., 2016). The mushroom bodies may also be important for visual navigation in desert ants, Cataglyphis fortis (Stieb et al., 2010, 2012) and Cataglyphis noda (Grob et al., 2017), as well as foraging in honey bees (Farris et al., 2001; Lutz et al., 2012; Cabirol et al., 2018). In the fruit fly, Drosophila melanogaster, the mushroom bodies have been shown to distinctly segregate visual and olfactory sensory input (Vogt et al., 2016). A model looking at navigation of the desert ant, C. fortis, has shown that the mushroom body circuitry has the capacity to facilitate visual homing using snapshot matching (Ardin et al., 2016).

Many recent studies have focused on characterizing the role of central complex in spatial learning and memory and in navigation. In insects, the prominent midline structure comprising the ellipsoid body, fan-shaped body, protocerebral bridge, and noduli (Ito et al., 2014) has long been shown to be important for locomotor activity (Strauss, 2002; Strausfeld and Hirth, 2013). The central complex also plays an important role in the integrative behaviors such as visual orientation and spatial integration (Homberg et al., 2011). It plays a role in visual pattern memory during foraging behaviors of D. melanogaster (Liu et al., 2006; Wang et al., 2008), visual pattern recognition (Pan et al., 2009), and spatial learning and place memory (Neuser et al., 2008; Ofstad et al., 2011; Varga et al., 2017). Disrupting central complex processing in D. melanogaster impacts spatial learning and memory (Ofstad et al., 2011). The ellipsoid body, containing the ring neurons, are known to be important for the recognition of visual patterns (Seelig and Jayaraman, 2013), have been shown to respond to visual landmarks when available (Seelig and Jayaraman, 2015), and may provide the neural substrate for visual navigation (Dewar et al., 2017). Research is increasingly demonstrating the importance of the central complex in spatial behaviors and navigation, such as path integration and steering in the bee (Stone et al., 2017), internal representation of the heading in D. melanogaster (Kim et al., 2017; Turner-Evans et al., 2017). In another species of cockroach, B. discoidalis, the central complex has been shown to be important in coding head direction relative to both internal cues and landmarks (Varga and Ritzmann, 2016), in addition to context-dependent movement (Martin et al., 2015). When taking into account all of this research, there is evidence that suggest that the mushroom bodies, the central complex, and/or the integrity of the projections running through the central brain are essential to spatial learning and memory in many invertebrate species.

# CONCLUSION

The findings from the experiments presented here invite an interesting comparison between the spatial mapping in the cockroach and the parallel map theory of hippocampal function (Jacobs and Schenk, 2003; Jacobs, 2012). The parallel map theory proposes that the hippocampus encodes space with two mapping systems. One, the "bearing map," encodes space based on directional cues such as gradients. The other, the "sketch map," encodes space based on positional cues. While the findings from the cockroach demonstrate the possible existence of comparable spatial frames of reference, it remains an open question whether the cockroach is using the olfactory cue, specifically the gradient information provided by the odor plume as a spatial frame of reference and if this frame of reference is encoded in parallel with the visual snapshot. While behavioral comparisons of spatial mapping in the cockroach and in animals with a hippocampus, such as rats, are quite possible with the current learning experiments, attributing such mapping functions to any structure

of the insect brain, as they have been for the hippocampus is an interesting challenge. These findings demonstrating spatial learning and memory capabilities of the cockroach, and the large amount of research increasingly showing that there are many similarities in the neural underpinnings of navigation and spatial behaviors in insects and mammals (Seelig and Jayaraman, 2015; Varga and Ritzmann, 2016; Kim et al., 2017; Turner-Evans et al., 2017; Varga et al., 2017), deserve further investigation and should invite further comparisons between spatial learning in mammals and insects.

#### DATA AVAILABILITY

The raw data supporting the conclusions of this manuscript will be made available by the authors, without undue reservation, to any qualified researcher.

### REFERENCES


### AUTHOR CONTRIBUTIONS

DL conceived, designed and performed the experiments, analyzed the data, and wrote the manuscript. MP performed the experiments, analyzed the data, and provided text and figures for the manuscript.

#### ACKNOWLEDGMENTS

The authors thank Sheyla Aucar and Austin Lawless for assistance with behavioral tests and the reviewers for the valuable feedback throughout the review process. Portions of the content presented here first appeared in the doctoral thesis of DL and were performed in the lab of Nicholas J. Strausfeld at the University of Arizona. The thesis is archived online and can be accessed through the UA Campus Repository.



**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2018 Pomaville and Lent. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Does Holistic Processing Require a Large Brain? Insights From Honeybees and Wasps in Fine Visual Recognition Tasks

Aurore Avarguès-Weber<sup>1</sup> \*, Daniele d'Amaro<sup>2</sup> , Marita Metzler<sup>3</sup> , Valerie Finke<sup>1</sup> , David Baracchi<sup>1</sup> and Adrian G. Dyer4,5

<sup>1</sup> Centre de Recherches sur la Cognition Animale, Centre de Biologie Intégrative (CBI), Université de Toulouse, CNRS, UPS, Toulouse, France, <sup>2</sup> Institut für Zoologie III (Neurobiologie), Johannes Gutenberg Universität Mainz, Mainz, Germany, <sup>3</sup> Department of Anatomy II, University of Cologne, Cologne, Germany, <sup>4</sup> School of Media and Communication, Royal Melbourne Institute of Technology, Melbourne, VIC, Australia, <sup>5</sup> Department of Physiology, Monash University, Clayton, VIC, Australia

#### Edited by:

Jeffrey A. Riffell, University of Washington, United States

#### Reviewed by:

Lesley J. Rogers, University of New England, Australia Cinzia Chiandetti, University of Trieste, Italy

\*Correspondence: Aurore Avarguès-Weber aurore.avargues-weber@univ-tlse3.fr

#### Specialty section:

This article was submitted to Comparative Psychology, a section of the journal Frontiers in Psychology

Received: 30 March 2018 Accepted: 09 July 2018 Published: 31 July 2018

#### Citation:

Avarguès-Weber A, d'Amaro D, Metzler M, Finke V, Baracchi D and Dyer AG (2018) Does Holistic Processing Require a Large Brain? Insights From Honeybees and Wasps in Fine Visual Recognition Tasks. Front. Psychol. 9:1313. doi: 10.3389/fpsyg.2018.01313 The expertise of humans for recognizing faces is largely based on holistic processing mechanism, a sophisticated cognitive process that develops with visual experience. The various visual features of a face are thus glued together and treated by the brain as a unique stimulus, facilitating robust recognition. Holistic processing is known to facilitate fine discrimination of highly similar visual stimuli, and involves specialized brain areas in humans and other primates. Although holistic processing is most typically employed with face stimuli, subjects can also learn to apply similar image analysis mechanisms when gaining expertise in discriminating novel visual objects, like becoming experts in recognizing birds or cars. Here, we ask if holistic processing with expertise might be a mechanism employed by the comparatively miniature brains of insects. We thus test whether honeybees (Apis mellifera) and/or wasps (Vespula vulgaris) can use holistic-like processing with experience to recognize images of human faces, or Navon-like parameterized-stimuli. These insect species are excellent visual learners and have previously shown ability to discriminate human face stimuli using configural type processing. Freely flying bees and wasps were consequently confronted with classical tests for holistic processing, the part-whole effect and the composite-face effect. Both species could learn similar faces from a standard face recognition test used for humans, and their performance in transfer tests was consistent with holistic processing as defined for studies on humans. Tests with parameterized stimuli also revealed a capacity of honeybees, but not wasps, to process complex visual information in a holistic way, suggesting that such sophisticated visual processing may be far more spread within the animal kingdom than previously thought, although may depend on ecological constraints.

Keywords: Apis mellifera, configural processing, face recognition, hierarchical stimuli, holistic processing, hymenopterans, Vespula vulgaris, visual cognition

# INTRODUCTION

fpsyg-09-01313 July 29, 2018 Time: 16:41 # 2

Humans and other primates have a remarkable ability to detect and visually identify conspecifics on the basis of their faces, which is a crucial capacity in our social interactions (Kanwisher et al., 1997; Pascalis et al., 2002; Wilmer et al., 2010; Young and Burton, 2017). A key mechanism of human face processing is that the visual system does not only use salient elemental features like hair, eyes, nose, or mouth to enable recognition, but it is rather the relationships between features or the configuration of a face that potentially allows for the seemingly advanced ability of humans to recognize conspecific faces (Carey and Diamond, 1977; Tanaka and Sengco, 1997; Collishaw and Hole, 2000; Maurer et al., 2002; Peterson and Rhodes, 2003).

Relationship processing between elemental features, a cognitive ability known as configural processing in visual cognition field, is considered to improve visual recognition accuracy. Three plausible levels of configural processing for face stimuli have been defined based upon human psychophysics experiments and/or neurophysiological recordings (Maurer et al., 2002). These three levels include (i) sensitivity to first-order relations where the spatial relationships between elemental features are processed (e.g., detecting a face because its features comprise a uniformed arrangement in which eyes are located above the nose which is located above a mouth); (ii) holistic processing, in which elemental features are bound together into a gestalt, and (iii) sensitivity to second-order relationships, in which slight variations of distances between features are perceived. Access to the first level of proposed processing is evidenced for example by a capacity to detect faces amongst considerable background noise like inverted two-tone Mooney faces (Maurer et al., 2002) and allow us to categorize stimuli as faces therefore activating specialized brain areas and specific holistic processing (Kanwisher, 2000; Maurer et al., 2002). Experimental access to holistic processing is achieved using stimuli manipulations including the part-whole effect and the composite-face effect (Carey and Diamond, 1977; Tanaka and Sengco, 1997; Collishaw and Hole, 2000; Maurer et al., 2002; Peterson and Rhodes, 2003). Indeed, because upright faces engage holistic processing, it is difficult to extract individual feature information separately. Thus, it is harder to recognize part of a face (e.g., the eyes) when perceived in isolation while the performance is restored when these features are replaced in the context of the full face (Part-Whole effect). Additionally, the creation of a composite face with features from different faces disrupts feature recognition as the composite face is processed holistically as a novel face (Composite face effect) (Carey and Diamond, 1977; Tanaka and Sengco, 1997; Collishaw and Hole, 2000; Maurer et al., 2002; Peterson and Rhodes, 2003). It is then often assumed that holistic representations enable second-order relationship processing that promotes reliable recognition among highly similar faces (Farah et al., 1998; Maurer et al., 2002; McKone et al., 2007; Taubert et al., 2011). Interestingly, it has also been suggested that holistic processing may operate as a general mechanism to aid reliable recognition from other competing objects in a complex visual environment (Tanaka and Gauthier, 1997; Farah et al., 1998; McKone et al., 2007; Taubert et al., 2011). Indeed, whilst the human and primate brain does have dedicated neural circuitry involved in face processing like the fusiform face area (Kanwisher et al., 1997; Kanwisher, 2000; Tsao et al., 2006), such areas do also facilitate recognition of other non-face stimuli when subjects are experts (Gauthier and Tarr, 1997; Gauthier et al., 2000).

Recently, the question on whether animals with different neural architecture may be able to process faces has received increased interest. There is growing evidence that animals including non-human primates (Sugita, 2008; Parr, 2011), dogs (Huber et al., 2013), sheep (Kendrick et al., 2001; Morton et al., 2018), magpies (Lee et al., 2011), house sparrows (Vincze et al., 2015), or fish species (Levey et al., 2009; Siebeck et al., 2010; Newport et al., 2016; Wang and Takeuchi, 2017) can reliably process images of human faces despite having very different neural architectures, and in many cases no shared evolutionary history to enable experience at viewing human faces [see Leopold and Rhodes (2010) for a review]. However, only a few studies studied the existence of holistic processing of conspecific or human faces in animals (Burke and Sulikowski, 2013). In parallel, the question of configural/holistic processing for other visual objects has been mainly investigated by using Navon-like hierarchical stimuli (stimuli showing a global shape or configuration created by the spatial arrangement of local shapes). Most tested species demonstrated a preference to process local information rather than the global configuration [e.g., baboons (Fagot and Deruelle, 1997), capuchin monkeys (Truppa et al., 2017), or chicks (Chiandetti et al., 2014)]. To date, only Humans (Navon, 1977), a fish species Xenotoca eiseni (Truppa et al., 2010) and honeybees (Avarguès-Weber et al., 2015) showed consistent global preference suggesting a general importance of visual configural processing in these species.

In this context, some social insects species became promising models of visual configural processing due to experimental access combined with evidence of impressive visual recognition abilities including face processing of conspecifics (Tibbetts, 2002; Sheehan and Tibbetts, 2011), human faces (Dyer et al., 2005; Dyer and Vuong, 2008; Avarguès-Weber et al., 2017), or configural processing of parameterised visual stimuli (Avarguès-Weber et al., 2010b, 2015; Howard et al., 2017). Thus, a paper wasp species (Polistes fuscatus) was shown capable of individual recognition of conspecifics (Tibbetts, 2002). In a follow-up study (Sheehan and Tibbetts, 2011), the recognition ability of P. fuscatus foundresses was evaluated for visual stimuli including conspecific faces, prey items, complex geometric shapes, or conspecific faces where configuration had been manipulated. P. fuscatus wasps' recognition level for conspecific faces was superior to all other stimuli in particular faces with altered configuration (Sheehan and Tibbetts, 2011). This evidence from P. fuscatus wasps shows that individual recognition via subtle visual discrimination is also possible in insects with potential convergence of visual strategies based on configural processing with mammals (Avarguès-Weber, 2012; Chittka and Dyer, 2012). Further works on wasps suggest that face recognition may have evolved several times in insects depending upon ecological constraints (Baracchi et al., 2015, 2016).

The fact that paper wasps could recognize conspecifics (Tibbetts, 2002) also lead to research testing whether honeybees might be able to recognize human faces (Dyer et al., 2005). When trained in an appetitive-aversive differential conditioning protocol to discriminate pictures of human faces chosen from a standard face recognition test as difficult to differentiate for human subjects (Warrington, 1996), free-flying honeybees could reliably recognize the rewarded target face even in the presence of very similar and novel distractor faces (Dyer et al., 2005). Subsequent work showed that honeybees could interpolate information from multiple viewpoints of faces to enable face recognition at novel viewpoint angles (Dyer and Vuong, 2008), or use configural mechanisms to enable first order processing of face stimuli (Avarguès-Weber et al., 2010b). Finally, in a recent experiment free flying wasps Vespula vulgaris were shown also capable to learn the same human faces pictures with performance similar to that of honeybees (Avarguès-Weber et al., 2017).

In the current study, we employ the framework for configural face processing proposed by Maurer et al. (2002) to test the capacity of both the honeybee (Apis mellifera) and the wasp (V. vulgaris) to process greyscale pictures of human faces used in previous studies (Dyer et al., 2005; Avarguès-Weber et al., 2017) as well as Navon-like geometrical hierarchical stimuli using a holistic processing mechanism. These visual objects, classically used in visual cognition studies, were chosen because of their complexity offering better chance to require configural processing to resolve them. In addition, the high perceptual difference between both types of pictures allows investigating whether holistic processing could be a general mechanism. Both of these insect species are visually active foragers, but neither has any evolutionary history of using visual information for recognition of human faces. We employ adaptations of the partwhole effect, and the composite-face effect experiments typically used to evaluate face processing in humans. Importantly, our study does not directly attempt to make inferential analyses between insect and human species, but seeks to understand whether our test model species show evidence of holistic-like processing in an attempt to gain insights into whether holistic processing is a mechanism that is general to visual systems in nature for fine discrimination.

# MATERIALS AND METHODS

#### Experiment 1: Human Faces Pictures

Experiments were conducted in 2013 at Mainz University with individually tagged and tested honeybees (A. mellifera L.) and wasps (V. vulgaris) trained by providing sucrose rewards to freely visit the experimental apparatus, a 50 cm diameter vertical screen which could be rotated to vary the spatial arrangement of the stimuli presented on it (Dyer et al., 2005; Dyer and Vuong, 2008) (**Figure 1A**). Only one individual was present at a time at the apparatus during the training and the tests. Two achromatic human faces from a standard face recognition test (Warrington, 1996) and used previously to investigate human face recognition abilities in bees (Dyer et al., 2005) and wasps (Avarguès-Weber et al., 2017) were chosen as complex visual stimuli to be discriminate. Four stimuli (two identical S+ and two identical S− stimuli; **Figures 1A,B**) were presented simultaneously on top of landing platforms offering a 10 µL drop of either a 25% (vol/vol) sucrose solution (S+) or a 60 mM quinine hemisulfate solution (S−), which promotes enhanced visual discrimination performances (Avarguès-Weber et al., 2010a). The reinforcement contingency was balanced between tested subjects. The face stimuli were attached on freely rotating 6 cm × 8 cm hangers that could be positioned in a number of random spatial positions and rearranged during the training by a rotation of the whole screen or manual displacements of the hangers (**Figure 1A**).

Before returning to the nest to deliver the sucrose collected, the bees or wasps typically made four to six choices (landing on a stimulus platform). Training length was chosen after pilot experiments to assure both species obtained a high level (≈80% of correct choices) of discrimination between the training faces, and a capacity to identify the target when presented with the inner part only of the training faces (Inner Test, see description of the tests below) consistent with previous evidence reported in Avarguès-Weber et al. (2010b). We thus used a training length of 180 choices for each bee, and 90 choices were necessary to reach a similar level of performance with the wasps. However, an inferential interpretation of the effect of training length between species was not a goal of the current study. In particular, experiments with bees and wasps were not conducted in parallel and may therefore have been subjected to differential seasonal effects for example. In this regard, our pilot tests found wasps only reliably forage for sucrose solution in the last weeks of summer which induces very limited experimental opportunity to test this species in free-flying conditions. Stimuli and landing platforms were washed with ethanol between foraging bouts and before the tests.

After training was completed, three non-reinforced test conditions were presented to the bees and wasps in which the first 20 choices were recorded (**Figure 1B**). The different test sessions were intermingled by three refreshing foraging bouts with the training conditions to maintain motivation. First, a Learning test presenting the training stimuli allowed accessing S+/S− discrimination level after the training session (**Figure 1B**). We then analyzed as a control the capacity of bees and wasps to discriminate both training face stimuli when only the stimuli inner parts were available (Inner test; **Figure 1B**). The comparison of performance level between the Inner test and the Part-Whole Test in which the S+ face was presented against a composed face (S− inner part surrounding by S+ outer features) was used as an indicator of holistic processing in the tested animals (**Figure 1B**). Both the Inner test and the Part-Whole test could only be resolved by the discrimination of the S+ vs. S− inner parts. The only difference between either test is that the inner parts were replaced in the context of a full image in the Part-Whole test. Thus, if bees' or wasps' visual recognition systems are sensitive to the "part-whole" effect, performance of the Part-Whole test should be higher than performance of the Inner test in which inner stimuli features are presented in isolation.

Finally, the Composite test aimed to investigate a potential composite face effect by offering a choice between a composed stimulus (S+ inner part and the S− outer part) and the S− face

stimulus. Performance in this test should be lower than in the Inner test if the tested subjects were relying on holistic processing to solve the discrimination task.

# Experiment 2: Hierarchical Navon-Like Parameterized Stimuli

This experiment was conducted with individually tagged and tested honeybees (in 2012, Mainz University) and wasps (in 2017, Mainz University) trained to freely visit a Y-maze setup covered by an ultraviolet transparent Plexiglas ceiling (**Figure 2A**). The entrance of the maze led to a decision chamber, where the flying insect could choose between the two arms of the maze (**Figure 2A**). One stimulus was presented vertically on each back wall of the arms which were placed at 15 cm from the decision chamber (**Figure 2A**). Such a setup allows for a controlled viewing distance as choices are recorded when the insect leaves the decision chamber thus entering one arm of the Y-maze. The visual angle subtended by the stimuli at the decision point was consequently controlled so that both small local features and large global features of the hierarchical stimuli were easily perceived by the animals.

The training phase consisted of a differential conditioning task with two hierarchical compound stimuli including a 11 cm square composed by the spatial arrangement of 12 repetitions of 1-cm up-triangles and a 11 cm diamond (45◦ rotated square) composed by 12 repetitions of 1-cm down-triangles (**Figure 2B**). For each tested subject, one of these stimuli was set in a balanced design as the S+ and associated with a 25% sucrose solution while the other was set as the S− and associated with a quinine solution (60 mM). Solutions were delivered in the center of each stimulus by means of transparent micropipettes. Between each foraging bout, the respective side of the S+ and the S− was allocated to the left or the right arm of the maze in a pseudo random fashion (e.g., the same stimulus was not presented in the same side more than twice in a row). If the subject chose the arm in which the S+ was presented, it could drink the sucrose solution ab libitum before returning to the nest. If the subject chose the S− arm, it was allowed to taste

the quinine solution and then to fly back freely to the alternative arm where it could drink the sucrose solution; but only the first choice, recorded when the animal entered an arm, was counted. The training lasted 36 choices which correspond to 36 foraging bouts in this setup. This training length assured similar level of performance both for the bees and the wasps.

After training was completed, the subjects faced a Learning test with fresh S+ and S− stimuli (**Figure 2B**). Then four different non-reinforced transfer tests were proposed in a random sequence order intermingled by three refreshing training bouts (**Figure 2B**). During the tests, contacts with the surface of the stimuli were counted for 45 s.

As a control, global feature learning was assessed by analyzing the insects' capacity to recognize the S+ global shape (square or diamond) vs. the S− global shape when presented in isolation, i.e., in the absence of the local features thus created by 1-cm wide plain lines (Global test; **Figure 2B**).

To evaluate the existence of the part-whole effect as indicator of holistic processing, we compared performance in the Global test to performance in the Part-Whole test offering a choice between the S+ global shape constructed by the S+ local elements (S+ stimulus) versus the S− global shape constructed also by the S+ local elements (composed stimulus S+/S−). In both tests, only the global information could be used as a cue but was presented in isolation in one case (Global Test) and in the whole context of a Navon-like stimulus in the other case (Part-Whole Test) (**Figure 2B**).

We then tested whether adding a novel local cue would impede recognition of the global cue (composite effect) in the Composite test (G+/Lnew vs. G−/Lnew) (**Figure 2B**). The performance in this test was also compared to the recognition level in the Global test where only global cues were available.

#### Statistical Analysis

Performances during the tests (proportion of correct choices out of the 20 test choices; a single value by subject) were analyzed with a generalized linear model (GLM) selecting a binomial distribution and a logit link function. This model only included the intercept term to test for a significant difference between the mean proportion of observed correct choices (p) and the proportion of choices expected by chance (p = 0.5). The stimulus set as rewarded (categorical factor) never had a significant influence on the performance (p > 0.05) and data were, therefore, pooled for the tests analysis. The performances of the different tests were compared with a GLMM in which individuals were considered as a random factor to account for

the repeated measurement design while the type of test was set as a categorical variable. The analyses were performed with R software, version 3.3.2 (R Development Core Team), lme4 package (Bates et al., 2014).

# RESULTS

#### Experiment 1: Human Faces Pictures Honeybees

Honeybees (N = 12) succeeded in learning the discrimination task between the two human faces (S+ vs. S−; **Figure 1C**). The discrimination performance was significantly higher than chance level in the non-reinforced Learning test where the bees had to choose between the S+ and S− stimuli [N = 12; 86.3 ± 2.6 (mean ± SEM) % of correct choices; GLM: z = 9.80, p < 0.001; **Figure 1C**]. There was no significant influence of the face used as S+ stimulus (z = 0.19, p = 0.85).

The bees were still capable of recognizing the training stimuli when only the inner parts were available (Inner test: 60.0 ± 3.8% of correct choices; z = 3.08, p = 0.002; **Figure 1C**). However, performance was significantly lower than for the whole faces (Inner test versus Learning test: GLMM: z = 6.29, p < 0.001; **Figure 1C**).

In the Part-Whole test, adding the S+ outer part to recreate whole faces allowed the restoration of the Learning test performance level although the bees could only rely as in the Inner test on the inner parts to discriminate both stimuli. Indeed, the outer parts were identical for both options (85.5 ± 2.6% of correct choices; z = 9.67, p < 0.001; comparison with the Learning test: z = 0.26, p = 0.79 and with the Inner test: z = 6.09, p < 0.001; **Figure 1C**). The honeybees seem thus sensitive to the "partwhole" effect as recognition of a part of the training stimulus was facilitated when presented in the context of a whole face.

When confronted to the Composite test in which the distractor (S−) outer feature was added to the inner part of the S+ face, the bees failed to recognize such composite stimulus as being more similar to the S+ face than the full S− alternative option (44.6 ± 5.6% of correct choices; z = 1.66, p = 0.09; **Figure 1C**). Results from this test suggest that honeybees are sensitive to the "composite-face" effect as they had greater difficulty to recognize the S+ inner feature when placed in the context of an incorrect whole face than presented in isolation (Composite test versus Inner test: z = 3.40, p < 0.001; **Figure 1C**).

#### Wasps

The wasps (N = 12) trained to discriminate the S+ and S− human faces successfully learned the task after 90 reinforced choices (77.9 ± 2.2% of correct choices in the Learning test; z = 8.10, p < 0.001; **Figure 1C**) and were able to use only the inner features of the faces to recognize the S+ stimulus (Inner test: 60.0 ± 2.5% of correct choices; z = 3.08, p = 0.002; **Figure 1C**) although performance level was significantly lower than with the whole face (Learning test versus Inner test: z = 4.20, p < 0.001; **Figure 1C**). There was no significant influence of the face used as S+ stimulus (z = 0.25, p = 0.80).

The wasps also showed restored performance when full faces were presented in the Part-Whole test even if the available information to solve the discrimination task remained the inner features only as for the Inner test (84.6 ± 3.5% of correct choices, z = 9.52, p < 0.001; Part-Whole test versus Learning test: z = 1.86, p = 0.06; Part-Whole test versus Inner test: z = 5.84, p < 0.001; **Figure 1C**). The wasps seem consequently also sensitive to the "Part-Whole effect" when extensively trained with complex visual stimuli.

Finally, in the Composite test, the wasps not only failed to recognize the S+ inner features when surrounded by the S− outer features ("Composite-face effect") but showed significant preference for the S− stimulus suggesting novelty aversion for the composed stimulus (37.1 ± 4.7% of correct choices; z = 3.96, p < 0.001; Composite test versus Inner test: z = 4.98, p < 0.001; **Figure 1C**). A similar tendency although not significant (44.6% of correct choices, p = 0.09; see above) was also observed in bees.

#### Experiment 2: Hierarchical Navon-Like Parameterized Stimuli Honeybees

Honeybees (N = 10) successfully learned to discriminate the S+ and S− hierarchical stimuli as performance in the Learning test was significantly above chance level (73.3 ± 2.7% of correct choices; z = 6.66, p < 0.001; **Figure 2C**). There was no significant influence of the stimulus used as S+ (z = 1.18, p = 0.24). The bees were capable to recognize the S+ global shape even when drawn with a solid line (interpolation) instead of distinct local features (Global test: 62.5 ± 2.6% of correct choices; z = 3.82, p < 0.001; **Figure 2C**) but this transformation resulted in poorer performance than in the Learning test (z = 2.43, p = 0.02; **Figure 2C**).

The bees behaved consistently with a sensitivity to the "partwhole effect" with parameterized stimuli as with the face stimuli: adding the same local features (L+) to the global shapes (Part-Whole test: G+L+ versus G−L+), thus re-constructing full hierarchical stimuli while still only offering the global information to allow solving the discrimination task, induced restored performance to a level similar to the Learning test performance (66.4 ± 2.5% of correct choices, z = 4.81, p < 0.001; Part-Whole test versus Learning test: z = 1.62, p = 0.11) although not significantly different from the Global test performance (Part-Whole test versus Global test: z = 0.82, p = 0.41; **Figure 2C**).

When facing the stimuli of the Composite test created by using novel local elements (dots), the bees failed to recognize the S+ and S− global features (49.7 ± 2.0% of correct choices, z = 0.20, p = 0.84; Composite test versus Global test: z = 2.85, p = 0.004; **Figure 2C**) thus suggesting again the influence of the "compositeface effect."

#### Wasps

The wasps (N = 6) trained to discriminate S+ from S− hierarchical Navon-like stimuli successfully solved the task as shown by their performance in the Learning test (68.0 ± 5.1% of correct choices, z = 3.11, p = 0.002; **Figure 2C**). There was no significant influence of the stimulus used as S+ (z = 0.55, p = 0.58). They were also capable of interpolating the learnt

stimuli to their global shape in the absence of local features (Global test: 71.3 ± 1.9% of correct choices; z = 2.52, p = 0.01; **Figure 2C**). Interestingly, removing local features did not impede wasps' performance (Global test versus Learning test: z = 0.44, p = 0.66; **Figure 2C**). A similar level of performance was obtained when the hierarchical structure was restored by adding the S+ local features to both global information (Part-Whole test: 68.0 ± 5.8% of correct choices; z = 2.20, p = 0.03; Part-Whole test versus Learning test: z = 0.73, p = 0.47; **Figure 2C**). The wasps also did not appear to experience difficulty in recognizing the global information when novel local features were used (Composite test: 66.5 ± 6.6% of correct choices, z = 4.25, p < 0.001; Composite test versus Learning test: z = 1.22, p = 0.22; **Figure 2C**). Thus, in this particular experiment, the wasps did not seem to use holisticlike processing mechanism to recognize simplified parameterized stimuli, in contrast to our results with honeybees.

# DISCUSSION

In this paper, we evaluated whether either of two hymenopteran species with relatively small brains of less than a million neurons might have a capacity for holistic processing of human faces, and parameterized stimuli, following the definitions for configural processing outlined by Maurer et al. (2002). Using the partwhole effect type experiment both honeybees and wasps showed a significant improvement to discriminate between inner features of the faces when they were shown together with identical outer features in a holistic stimulus than when presented in isolation (**Figure 1**). However, visual processing was totally disrupted when the correct face inner features were combined with the outer features of the distractor, showing that both bees and wasps were sensitive to the composite-face effect in their visual processing of stimuli (**Figure 1**). Thus, both bee and wasp species showed evidence consistent with holistic processing when having to recognize pictures of human faces, even though neither species has any ecological reason of having experience with human faces.

In the experiments with parameterized stimuli, honeybees also exhibited choice behavior consistent with holistic processing as performance was lower when bees had only access to the global features than when presented together with the local features (part-whole effect) and the bees' choices collapsed to chance level when the same global features were shown together with novel local features (composite effect) (**Figure 2**). However, in wasps, no change in the capacity to recognize global features was observed, neither when presented in isolation, in a whole hierarchical context, nor together with novel local cues (**Figure 2**). Wasps did not seem consequently to rely on holistic-like processing with these particular stimuli. Different hymenopteran species thus process and implement various forms of configural processing in different ways. Interestingly, honeybees are known to process Navon stimuli with a global preference consistent with configural processing, but this preference could be modulated with priming to local stimulus elements (Avarguès-Weber et al., 2015), showing evidence of plasticity in the application of visual processing rules by honeybees. In bees, the sensitivity to some contextual visual illusions also considered as dependent on configural processing could also be modulated and is in particular under the influence of the conditioning procedure (Howard et al., 2017). The influence of testing procedure might also be at the origin of the difference in Global/Local processing between species as the fish species (Truppa et al., 2010) and bees (Avarguès-Weber et al., 2015) were the only animals tested while having the possibility to move toward the stimuli thus promoting configural processing (Rosa Salva et al., 2014). Thus, differences in visual strategies between different hymenopteran species for specific stimuli may depend upon a variety of factors that remain to be characterized. As both species shared a similar visual system (compound eyes and brain structure) due to their phylogenetic common history, it could be speculated that the difference in the use of holistic processing may be dependent of ecological differences, for example, in foraging (prey for wasps; flowers for honeybees) either through evolutionary adaptation or individual experience. Despite this difference for parameterized stimuli, we did observe some evidence that both species, despite their miniature brain, can holistically process visual information. This result suggests therefore that configural processing could be a more widespread visual solution in nature, and it would thus be of value to explore such a capacity in a wider range of vision-dependent species to understand how environmental and neurobiological contexts may influence visual recognition strategies.

The fact that two hymenopteran species show some evidence of holistic-like processing of complex visual stimuli leads to the interesting question of where in the insect brain such a process may take place. We hypothesize that mushroom bodies, sharing analogies with the higher cortical centers of vertebrate brains (Farris, 2008) and believed to be strongly linked to learning and memory processed in arthropod brains (Hammer and Menzel, 1995; Mizunami et al., 1998; Strausfeld et al., 1998; Hourcade et al., 2010; Devaud et al., 2015), should be the first structures to test for their implication in configural processing. In addition, Hymenopteran species such as bees and wasps do possess particularly developed mushroom bodies in comparison to other insects (Farris, 2008). For instance, the calyces of the mushroom bodies are doubled and expanded while receiving novel afferences from the visual part of the brain in comparison to Drosophila mushroom bodies (Farris, 2008; Avarguès-Weber and Giurfa, 2013). As the evolutionary development of mushroom bodies started back with ancestral parasitoid wasps (Farris and Schulmeister, 2011) that shared with bees spatial, visual, or olfactory learning need, the mushroom bodies are consequently considered as promoting learning abilities and flexibility (Giurfa, 2003; Chittka and Niven, 2009).

Finally, our new findings fit with the framework proposed by Chittka and Niven (Chittka and Niven, 2009) that large brains may not be necessary for processing seemingly complex stimuli, like faces, but rather the ecological conditions may enable the capacity to develop a brain that can use sophisticated strategies (Chittka and Niven, 2009; Chittka and Jensen, 2011; Chittka and Dyer, 2012). It is nevertheless likely that this new work has just scratched the surface of how hymenopteran insects, or even other animals may use configural processing, and it will be necessary to explore the very wide range of approaches

applied in human psychophysics to build a more comprehensive understanding of these phenomenon in nature and in particular, how the impressive abilities of biological brains are possible, and what might be solutions that could be applied to machine vision (Kleyko et al., 2015; Cyr et al., 2017).

#### ETHICS STATEMENT

Our research involves honey bees and wasps that are not animal models for which approval of an ethical committee is required. A minimum number of animals were used to resolve our scientific question. The animals remained free during the whole experiment and were not harmed in our experimental procedure.

#### AUTHOR CONTRIBUTIONS

AA-W and AD conceived the study and designed the experiments. Dd'A, MM, VF, DB, and AD performed the

#### REFERENCES


experiments. AA-W analyzed the data. AA-W and AD wrote the manuscript.

#### FUNDING

Our work was supported by the Fyssen Foundation, the French National Research Center (CNRS), and the University Paul Sabatier of Toulouse. AD acknowledges the Australian Research Council DP0878968/DP0987989, the Alexander von Humboldt Foundation, and the USAF AOARD for assistance in developing this work.

# ACKNOWLEDGMENTS

We are grateful for advice and assistance provided by Dr. Jürgen Schramme, Dr. Jair Garcia, Professor Marcello Rosa, and Professor Roland Strauss to help facilitate this research and to Michael Merz for technical assistance.


memory trace in the insect brain? J. Neurosci. 30, 6461–6465. doi: 10.1523/ JNEUROSCI.0841-10.2010


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2018 Avarguès-Weber, d'Amaro, Metzler, Finke, Baracchi and Dyer. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# High-Speed Videography Reveals How Honeybees Can Turn a Spatial Concept Learning Task Into a Simple Discrimination Task by Stereotyped Flight Movements and Sequential Inspection of Pattern Elements

#### Marie Guiraud1†, Mark Roper 1,2† and Lars Chittka1,3 \*

#### Edited by:

Thomas Bugnyar, Universität Wien, Austria

#### Reviewed by:

Aurore Avargues-Weber, UMR5169 Centre de Recherches sur la Cognition Animale (CRCA), France Tohru Taniuchi, Kanazawa University, Japan

> \*Correspondence: Lars Chittka l.chittka@qmul.ac.uk

†These authors have contributed equally to this work.

#### Specialty section:

This article was submitted to Comparative Psychology, a section of the journal Frontiers in Psychology

Received: 21 March 2018 Accepted: 13 July 2018 Published: 03 August 2018

#### Citation:

Guiraud M, Roper M and Chittka L (2018) High-Speed Videography Reveals How Honeybees Can Turn a Spatial Concept Learning Task Into a Simple Discrimination Task by Stereotyped Flight Movements and Sequential Inspection of Pattern Elements. Front. Psychol. 9:1347. doi: 10.3389/fpsyg.2018.01347

<sup>1</sup> School of Biological and Chemical Sciences, Queen Mary University of London, London, United Kingdom, <sup>2</sup> Drone Development Lab, Ben Thorns Ltd, Colchester, United Kingdom, <sup>3</sup> Wissenschaftskolleg, Institute of Advanced Study, Berlin, Germany

Honey bees display remarkable visual learning abilities, providing insights regarding visual information processing in a miniature brain. It was discovered that bees can solve a task that is generally viewed as spatial concept learning in primates, specifically the concept of "above" and "below." In these works, two pairs of visual stimuli were shown in the two arms of a Y-maze. Each arm displayed a "referent" shape (e.g., a cross, or a horizontal line) and a second geometric shape that appeared either above or below the referent. Bees learning the "concept of aboveness" had to choose the arm of the Y-maze in which a shape–any shape–occurred above the referent, while those learning the "concept of belowness" had to pick the arm in which there was an arbitrary item beneath the referent. Here, we explore the sequential decision-making process that allows bees to solve this task by analyzing their flight trajectories inside the Y-maze. Over 368 h of high-speed video footage of the bees' choice strategies were analyzed in detail. In our experiments, many bees failed the task, and, with the possible exception of a single forager, bees as a group failed to reach significance in picking the correct arm from the decision chamber of the maze. Of those bees that succeeded in choosing correctly, most required a closeup inspection of the targets. These bees typically employed a close-up scan of only the bottom part of the pattern before taking the decision of landing on a feeder. When rejecting incorrect feeders, they repeatedly scanned the pattern features, but were still, on average, faster at completing the task than the non-leaners. This shows that solving a concept learning task could actually be mediated by turning it into a more manageable discrimination task by some animals, although one individual in this study appeared to have gained the ability (by the end of the training) to solve the task in a manner predicted by concept learning.

Keywords: active vision, Apis mellifera, cognition, feature detection, local features, video tracking, visual learning

# INTRODUCTION

Concept learning is often viewed as a key ingredient of what makes humans uniquely intelligent, since it appears to involve a number of mental abstractions (e.g., equivalence, area, volume, and numerosity) (Piaget and Inhelder, 1956; Marcus et al., 1999), as well as sentence constructions and mathematical operations (Edward et al., 1992; Chen et al., 2004; Christie et al., 2016). Yet, in the last 50 years, concept learning has been a recurrent theme when exploring animal cognition (Savage-Rumbaugh et al., 1980; Savage-Rumbaugh, 1984; Akhtar and Tomasello, 1997; Zayan and Vauclair, 1998; Depy et al., 1999; Penn et al., 2008; Shettleworth, 2010). Scientists have discovered concept learning in various animal taxa, for example the learning of sameness and difference concepts in the pigeon (Zentall and Hogan, 1974), in ducklings (Martinho and Kacelnik, 2016), monkeys (Wright et al., 1984), the honeybee (Giurfa et al., 2001), and one study comparing two species of monkeys and pigeons (Wright and Katz, 2006); other studies focused on oddity and non-oddity in monkeys (Moon and Harlow, 1955), pigeons (Lombardi et al., 1984; Lombardi, 2008), rats (Taniuchi et al., 2017), sea lions (Hille et al., 2006), dogs (Gadzichowski et al., 2016), and honeybees (Muszynski and Couvillon, 2015); the concept of symmetry/asymmetry in honeybees (Giurfa et al., 1996). Spatial concepts such as aboveness and belowness have been explored in a number of vertebrates (Zentall and Hogan, 1974; Depy et al., 1999; Spinozzi et al., 2004), and also the honeybee (Avarguès-Weber et al., 2011, 2012). However, the majority of studies have focused on whether or not the subject could solve a given task, not on how the animals actually solved them. Similar tasks might be solved by profoundly different mechanisms and behavioral strategies in different animal species.

In a typical protocol to explore potential concept learning animals, Avarguès-Weber et al. (2011) tested honeybees in a series of binary choices in a Y-maze flight arena to assess whether bees could master the conceptual spatial relationships of "above" and "below." The experimental paradigm consisted of a pair of stimuli, each with a variable geometric target shape located above or below a shape (e.g., a black cross) that acted as referent point (**Figure 1**). One of the arms of the Y-maze flight arena presented the target above the referent and the other presented the same target but below the referent. The bees had to learn that either the "above" or "below" pattern configuration was associated with reward (sucrose solution provided in the center of the stimulus wall), and the other pattern lead to a punishment (quinine solution). After fifty training bouts, bees were subjected to an unrewarded transfer test using novel target shapes to determine if they had learnt the concept of "aboveness" or "belowness." The results indicated that positive transfer occurred (Avarguès-Weber et al., 2011). These trials did not, however, show how bees solved the problem, and therefore a number of alternative hypotheses might potentially explain these results. Indeed, depending on how bees approach the task during training, they may evaluate their options and learn differently. Bees trained to an "aboveness" task could simply fly to referent (the invariant component of the display) and verify that the ventral visual field is empty (without examining the item above the referent). The reverse solution could be applied in a "belowness" task. Bees could approach a stimulus scanning only the top (or the bottom) shape and learn that they should either expect the referent in that position, or anything other than the referent (depending on whether they are learning "above" or "below"). Avarguès-Weber et al. (2011) proposed that bees evaluate the whole compound stimulus, using the relative position of the stimuli shapes to determine their "above" or "below" relationship, and then choose accordingly. It is also conceivable that different individuals use different strategies when faced with the same task, or indeed that the same individuals use different strategies in different phases of their training.

We tested these hypotheses to understand the bees' strategies in solving such tasks by replicating the original honeybee "above and below" experiments (Avarguès-Weber et al., 2011), but with the addition of high-speed cameras to record the flight paths during every training and transfer test trial. We aimed to determine how variations in the bees' behavior toward the stimuli during the training impacted their learning abilities, and subsequent transfer test performances. We evaluated which shapes or stimuli regions the bees inspected (including the order of elemental observations and repetitions) time spent in each activity, as well as performances during and after training.

# MATERIALS AND METHODS

#### Setting and Material

Experiments were conducted over three consecutive summers (2015–2017). Honeybees (Apis mellifera) from three colonies were allowed to collect 20% sucrose solution (w/w) from a gravity feeder located either 20 m or 2 m from the hives depending on the year. This type of feeder is shown in (Frisch (1965) his Figure 18)–it allows several dozen bees to feed simultaneously and commute between the hive and the feeder. This ensures that a good number of motivated foragers are typically available near the training setup. An individual bee is then tempted away from this communal feeding station by offering it a reward that is higher in quality than that of the gravity feeder. In our experiments, we offered a cotton bud, soaked with 50% sucrose solution (w/w) to a bee that had just landed near the gravity feeder. Once the bee walked on the cotton bud, and began feeding, she was slowly transferred by the experimenter to one of the feeding tubes within the apparatus. A small colored dot was applied to the bee's dorsal abdomen using colored Posca marking pens (Uni-Ball, Japan), while she was feeding. Upon the return from the hive, the bee was typically found back at the gravity feeder or near the setup. This procedure was repeated until the bee learnt to fly directly to the feeding tubes at the end of the Y-maze arms (the bee was put in either the left or right arm in a pseudo-random sequence, no more than twice in the same arm and usually needed a repetition of two to three of these operations before the task can be initiated). This method allowed us to limit the number of bees near the apparatus. Additionally, any unmarked bees were removed from the experimental area. Only one bee was trained at a time within the Y-maze, and we followed the original experimental protocol (Avarguès-Weber

et al., 2011), albeit with some modifications. The Y-maze (see **Figure 1**) consisted of an entrance hole that led to a central decision chamber, from which two arms extended. Each arm measured 40 × 20 × 20 cm (L × H × W). Within each arm, a moveable rear wall (20 × 20 cm) was placed 15 cm from the decision chamber, providing support for the stimuli and feeder tubes. Unlike the experiments by Avarguès-Weber et al. (2011), no Perspex transparent cover was placed on the top of the Ymaze flight arena; this was to allow for an unobstructed and undistorted view while taking high-speed video recordings. Two Yi (Xiaomi Inc. China) sport cameras were positioned side-byside 10 cm above the entrance of the Y-maze. Their field of view was adjusted such that they looked down into the arena at ∼60◦ from horizontal, establishing in both cameras a wide-angle view of both arms. Each Yi camera was configured to record at 120 fps (frames per second) at a resolution of 720 p (1,280 × 720 pixels). Once the bee entered the arena, both cameras were started, such that there was an individual video file per camera, per trial. Filming of a trial began when the honey bee entered the flight arena and continued until the bee entered the rewarding feeding tube.

Each stimulus was composed of black patterns on a 20 × 24 cm (W × H) white UV-reflecting paper, printed using a highresolution laser printer. The patterns were disposed of after a single use, to prevent odors being deposited by the bees and being subsequently used as an olfactory cue during learning. Another modification of the setup by Avarguès-Weber et al. (2011) was that we had to modify the feeding stations. In the earlier study, this was a tube that protruded into the arena, and was filled with sucrose solution from the side of the arena. We performed a pilot study, collecting high speed video footage of two bees and found that bees made brief antennal touches to the feeders during fly-bys, allowing them to assess whether they contained sucrose solution prior to the decision to land (**Supplementary Video 1**) (Such antennal contacts are so brief that they are practically undetectable to the naked eye or with conventional video footage). To prevent bees from such contacts, our visual stimuli were combined with a centrally located feeding

tube (1 × 0.5 cm) that led to 50% sucrose solution (w/w) (see **Figure 1** for protocol). This was implemented to prevent sucrose solution being deposited on the entrance of the feeding tube during refilling, thereby forcing the honeybees to crawl into the tube (or at least land and put the head in the tube, see **Supplementary Videos 2**–**5**) to determine if it contained a reward. These feeding tubes were cleaned between trials, again to prevent odor cues being used in subsequent trials. Blank brown cardboard cover-plates 20 × 20 × 0.5 cm were placed in front of each of the two stimuli to prevent a bee from seeing the patterns or accessing the feeding tubes before a trial had begun. Two pairs of achromatic patterns were presented during each trial.

#### Phase 1–Pre-Training

For Phase 1 pre-training, the pair of stimuli consisted of blank white paper for one arm, and in the other white paper with a black cross (4 × 4 cm), which was later used as the "referent" in training (**Figure 1**). Each individual bee was first trained using an absolute conditioning protocol (Giurfa et al., 1999) in the Y-maze with the rewarding pattern presented in each arm in a pseudorandom sequence. In this, we followed the published protocol of Avarguès-Weber et al. (2011), where such a pseudorandom choice of the Y-maze arm was also reported. The rewarding stimulus was always a black cross randomly positioned on the white background. The other arm of the maze contained a fresh blank white sheet of paper (unpaired stimulus) with the feeding tube providing an aversive quinine solution. The bee's first choice (e.g., the bee touching and entering the feeder) was recorded and acquisition curves produced by calculating the frequency of correct choices per block of five trials. After 15 training trials, a discrimination test was introduced. In this test, two patterns were used; one consisted of the familiar cross, and the other contained one of five alternative shapes (to be used as targets in later training: concentric diamonds (5 × 6 cm), a small horizontal bar (1 × 3 cm), a vertical grating (5 × 5 cm), a filled circle (3 cm in diameter), or a radial three-sectored pattern (4 × 4 cm) (Avarguès-Weber et al., 2011). Neither pattern was rewarding, with both feeding tubes leading to 30 µl of water. The bee entered as normal but was given 45 s to explore the new configuration. The number of visits to each feeding tube was recorded.

# Phase 2–Main Training

In the main training phase, bees that completed phase 1 pretraining were either subjected to an "above," or a "below" differential conditioning protocol (see **Figure 1**). Each stimulus contained a pair of shapes. One was the same cross as used during pre-training, and which was now the "referent," being present in all stimuli. The other shape was a geometric "target" shape which could be either concentric diamonds, a small horizontal bar, a vertical grating, a filled circle, or a radial three-sectored pattern (Avarguès-Weber et al., 2011). These target shapes were horizontally aligned with the cross (either above or below it) and the pair of shapes ("referent" cross and "target") positioned randomly on the paper (centered, top-left, bottom-right, etc.). Two stimuli were presented in each trial, one pair in each of the Y-maze arms. Both stimuli contained the referent cross shape and another shape selected from the four available target shapes (excluding the shape used for that bee's phase 1 discrimination test). When the bee was assigned to the "above" group (Group A) she had to learn that the rewarding pattern would be the stimulus where the target would appear above the referent cross, and this arrangement would be associated with ad libitum 50% sucrose solution (w/w). The other stimulus (CS– or negative conditioned stimulus) presented the target below the cross and its feeding tube led to saturated quinine solution (Group B, "below" bees were trained with the reciprocal stimulus being aversive). If the bee chose the CS–, it tasted the quinine solution, and was allowed to continue flying within the flight arena inspecting the patterns until it discovered the rewarding feeder. The CS+ and CS– stimuli were presented in a pseudo-random sequence (never more than two consecutive trials on the same arm, see **Figure 1**) to prevent the bees, as far as possible, from learning a side preference. After feeding, the bee would depart for the hive, and return approximately every 3–10 min. This interval allowed for the next pairs of stimuli to be inserted into the Y-maze. The bees were trained for 50 trials. The first feeder choice was recorded upon the bee entering the maze after returning from the hive. Acquisition curves were produced by calculating the frequency of correct choices per block of 10 trials. Following the last acquisition trial, non-rewarded tests were performed with novel stimuli (utilizing the 5th geometric shape excluded from the training trials). During the tests, both the first feeder choice and the cumulative contacts with the feeders were counted for 45 s. The choice proportion for each of the two test stimuli was then calculated. Each test was performed twice, interchanging the sides of the stimuli to control for side preferences. Three rewarding trials using the training stimuli were conducted between the tests to ensure that foraging motivation did not decay owing to non-rewarded test experiences.

#### Video Analysis

The videos for each of the 50 training trials, for each bee, were replayed on a computer monitor in slow motion (1/8th of the regular speed) so that the particular flight trajectories of the bee could be observed and annotated. We analyzed 46h of raw footage (368 in slow motion) of videos to create the dataset. The honeybees typically displayed three types of flight characteristics during a trial: (a) direct flights: in these instances, the bees would enter the flight arena and fly directly to one or other of the feeding tubes (**Supplementary Video 2**). These flights would take less than a second until the bee had landed on the feeding tube, (b) scanning behavior: here the bees would either briefly fly toward one of the pattern shapes (0.5–2.0 s, brief inspection; **Supplementary Video 3**) or scan the shape with slow horizontal movements, repeated several times, with a typical duration between 1 and 15 s (**Supplementary Video 4**); (c) repetitive scans after a wrong decision: bees would successively scan feeder, top shape and bottom shape a number of times before changing arm (**Supplementary Video 5**).

Our video analysis focused on recording the following types of behaviors: side preference (upon entering the Y-maze, whether the bee displayed a consistent preference for the left or right arm of the apparatus when first selecting an arm during a trial); correct arm choice (if the bee initially selected the arm that contained the correctly configured pattern or CS+ arm), direct flights (if the bee flew directly to a feeder without scanning the patterns, recorded for both CS+ and CS– arms), and all scanning points (which component of the pattern the bee visited (bottom shape, top shape, and center (feeder). This included both scanning behavior and the less common brief inspections of shapes. A bee was designated as a "learner," if during the last 20 trials of complete training, it achieved an average of at least 60% correct choices and had at least 70% correct choices in one block of these two blocks of 10 trials. Otherwise it was classified as a non-learner bee (see **Figure 2**). Performance of balanced groups during acquisition was compared using Kruskal–Wallis H tests, and statistics within groups and trial blocks were tested using Mann–Whitney U tests, as well as tests against chance. All statistics were calculated using Python programming language.

#### RESULTS

#### Training and Tests Performance

Each bee took between 8 and 16 h to complete the training and testing phases. Thirty-seven bees that commenced training failed to complete the full protocol (either the bee did not return to the experiment after a trial, or poor weather conditions interrupted the bees' foraging). In total, 21 honey bees were trained. Two were excluded because they were mistakenly exposed to three or more rewarding patterns on the same Ymaze arm (A6 and B7, see **Supplementary Figures 1**, **2** for individual data). Of the remaining 19 bees, 9 bees were trained on the "above" protocol (Group A bees) and 10 bees on the "below" protocol (Group B bees). Seven of the ten Group B bees were successful at learning their task (correct stimulus having target shapes below the crosses). In contrast, only four individual bees from Group A learnt to identify the patterns with the target shapes above the crosses (**Figure 2**). Thus, in total eight bees (42%) failed to learn the task in our experiments; this contrasts with previous experiments (Avarguès-Weber et al., 2011) where all bees were reported to solve the task. Unless otherwise indicated, groups from the "above protocol" and "below protocol" were pooled, as there were no significant differences between them (these non-significant results are given in **Supplementary Tables 1**–**7**). **Figure 2** shows a summary of these results grouped into the learner and non-learner bees (Individual results: **Supplementary Figures 1**, **2)**. Overall, the bees we had classified as learners exhibited training performances which improved over time [Kruskal–Wallis H(2) = 10.5; df = 10; P = 0.03] while non-learners did not [Kruskal–Wallis H(2) = 3.7; df = 7; P = 0.454]. Moreover, the learner group selected the correct feeder 61% of the time, with bees averaging 66.4% over the last 10 trials; these bees performed significantly better than chance over each of the last three blocks of 10 trials (Mann– Whitney U for learners' training: df = 10; trials 21–30: U = 16.5, P = 0.004; trials 31–40: U = 16.5, P = 0.004 and trials 41–50: U = 16.5, P = 0.004; **Supplementary Table 1**). The non-learner group of bees, on the other hand, selected the correct feeder 44% of the time, with bees averaging 48.8% over the last 10 trials. These bees did not perform significantly better than chance over each of the last three blocks of 10 trials (Mann–Whitney U for learners training: df = 7; trials 21–30: U = 20, P = 0.226; trials 31–40: U = 16, P = 0.103 and trials 41–50: U = 28, P = 0.711; **Supplementary Table 1**).

During the transfer tests, bees were presented with stimuli using a novel target shape, above or below the familiar referent crosses. The learner group exhibited a preference for the correct stimulus during transfer tests with 63.6% (Mann–Whitney U for correct choices during test above chance for learners: U = 38.5, df = 10; P = 0.045; **Supplementary Table 2**). Similar results were seen for the average percentage of correct touches over the 45 s tests (58.8%; Mann–Whitney U test–choices for correct stimulus above chance for learners: U = 22.0, df = 10; P = 0.00; **Supplementary Table 2**). Although statistical analysis shows significance for correct choices, individual bees differed widely in the investigation of unrewarded stimuli, and also in terms of performances according to the sequence of the tests (first or second unrewarded test; see **Supplementary Figures 1**, **2**). The non-learner group of bees did not perform any better than chance, achieving just 43.8% (Mann–Whitney test U = 28.0; df = 7; P = 0.335; **Supplementary Table 2**) for first

choice during tests, and 51.2% (Mann–Whitney test U = 16.0, df = 7; P = 0.173; **Supplementary Table 2**) correct percentage of accumulative touches over 45 s, respectively (**Figure 2**).

In earlier works it was reported that bees were able to solve the task by using the spatial configuration of the elements of the stimuli (e.g., the target in relation to the referent) when viewing both patterns from the decision chamber, and choosing a Y-maze arm accordingly (Avarguès-Weber et al., 2011, 2012). In our study, we found that 13 of the 19 bees that completed training exhibited a strong side preference when entering the setup (a choice of left or right arm of ≥70%). Unsurprisingly perhaps, given the widespread nature of side biases, bees of the learner group did not choose the correct arm of the Ymaze significantly more than chance (Mann–Whitney U = 33.0, df = 11; P = 0.077; **Supplementary Table 3**). However, given this significance level we cannot reject with certainty the possibility that these bees initiated their decision making process in the decision chamber, and tended to do so correctly. Indeed, a single individual managed 90% correct choices from the decision chamber in the final 10 visits of training. This individual had already had above average performance throughout training (when making decisions close up to the target area) and appeared to switch strategies near the end of training so that choices were now initiated in the decision chamber (**Supplementary Figure 2**, bee: B8).

However, learner bees as a group failed to reach significance in choosing the correct Y-maze arm. We then evaluated the decision making process once bees had entered the arms of the Y maze. We first asked if the initial (accidental or via side bias) selection of the correct arm led to the choice of the correct feeder. Bees in the learner group selected the rewarding feeder more than 94.6% of the time after initially having entered the correct Y-maze arm, leading to no difference between the number of times they chose the correct arm, and the number of times they chose the correct feeder after choosing the correct arm (Mann–Whitney U = 42, df = 10, P = 0.238, no difference, thus high similarity; **Supplementary Table 3**). This behavior was also observed in the non-learner groups (Mann–Whitney U = 23.0, df = 7, P = 0.373; **Supplementary Table 3**) (**Figure 2**). However, the learner group of bees showed an ability to revert an incorrect first choice of a Y-maze arm during training by inspecting the stimulus but subsequently choosing to go to the other arm and select the feeder there. When the individuals of the learner groups entered an incorrect arm, they abandoned the arm a total of 48 out of the 252 incorrect choices (19%), and an average of 28.3% of such occurrences during the last 10 trials. This significantly differed from the non-learner group, which only left the wrong Y-maze arm 14 out of 228 wrong arm visits (6.1%) (Mann–Whitney U, difference between learner and non-learner groups U = 2.0, df = 18, P < 0.001; **Supplementary Table 4**).

# Spatial Conceptual Learning or Discrimination Task?

Having shown that a subset of our bees (learners from both the "above" and "below" groups) solved their respective tasks, we used the high-speed video recordings captured during each trial to analyze the sequential choices of both learner and non-learner group of bees during training. Avarguès-Weber et al. (2011) suggested that bees could use the spatial relationship between the two shapes present in the stimulus to solve the task. In this condition, bees would need to either make their decisions at some distance from the patterns (i.e., from the decision chamber), or by sequentially inspecting the two shapes within a pattern before choosing one of the feeders.

However, upon entering a Y-maze arm, bees did not fly directly to a feeder but typically spent time scanning the stimulus in the selected Y-maze arm. Interestingly, in all conditions below, no significant differences were found between learners and non-learners, thus both groups were pooled (**Supplementary Tables 5**–**7**). For analysis, three options were considered: bees could go directly to the feeder (**Supplementary Video 2**), scan the bottom shape (**Supplementary Video 4**), or the top shape. In all cases, chance represents 33.3% (50 trials and three options).

In all bees, the first item scanned was the bottom shape of the stimulus, in 64.2% of the cases (bottom choice vs. chance (33.3%) Mann–Whitney U = 0.0, df = 18, P = 0.0; **Supplementary Table 5**). The remaining 35.8% were split between feeder and top item. Collectively, in just 22.2% of flights did bees fly directly to a feeder. The majority of the direct flights to a rewarding feeder were by the learners (65.3%), constituting 10.7% of their trials. Similarly, 9.8% of learner bee flights were directly to the wrong feeders. Flying directly toward the top shape of the stimulus occurred in only 13.7% of total trials (**Figure 3**).

We additionally analyzed how each group of bees made use of targets and referents (see **Supplementary Figures 3**, **4** and **Supplementary Data Sheets 1**, **2**). However, this analysis only confirmed that bees from all groups have a strong preference for scanning the bottom item first (independent of whether it was a target or cross shape). Bees did not usually choose a feeder as their first location approached (as one might expect if the decision had been arrived at in the decision chamber of the Y-maze). Even if the arrangement of items in a stimulus was analyzed only by close-up scanning to solve the task, the logical following choice would be to scan the top item after the initial inspection of the bottom item. Yet, of the three options (top, bottom item and feeder) the second inspection point for any bee would often be one of the two feeders, in 58.1% of the cases (difference from a chance expectation of 33.3%–Mann–Whitney U = 0.0, df = 18, P = 0.0; **Supplementary Table 6**) although learners appear to choose feeders as second scanning item less (56% on average) than bees of the non-learner group (64.1% on average).

# Differences Between Learners and Non-learners

To explore the causes of differences in performance between learners and non-learners, we evaluated the number of items scanned by the bees and the intervals between entering the setup, scanning items, and selecting a feeder.

Over the entire 50 training bouts, the average cumulative number of scanning behaviors by each bee was 375.3 (±60.8) (minimum: 265; maximum: 523). Learners tended to display fewer inspections overall (362.5 ± 59.3) than non-learners (392.9 ± 62.4) but there was pronounced individual variation and therefore no significant difference between groups (learners vs. non-learners: Mann–Whitney U = 167.0, df = 18, P = 0.39; **Supplementary Table 7**). Interestingly, more inspections were made by learners (98.1 ± 23.8) than non-learners bees (77.6 ± 54.6) before making a correct choice (Mann–Whitney U = 20, df = 19, P = 0.026; **Supplementary Table 7**). Learners approached and scanned another item than the feeding tube in 96% of the cases before making a correct choice vs. 45% for non-learners, which, in in two-third of the cases would be the lower item of the stimulus. Moreover, non-learners displayed slightly more inspection behavior (315.3 ± 90.3 items inspected) than learners (264.5 ± 63.3) when making an incorrect decision but this difference is not significant (Mann–Whitney U = 28, df = 18, P = 0.1; **Supplementary Table 7**). For learners and non-learners, the number of scanning behavior increased strongly after an incorrect choice (by a factor of 2.7 for learners and 4.1 for nonlearners). When making an incorrect choice, after first probing the quinine solution, the bee will typically exhibit a repetitive sequence of scanning behaviors of the feeder, the top and bottom shape of the stimulus a multiple times before departing to the opposite arm of the Y-maze). The number of items scanned was 9.8 on average and ranged from 1 to 47. Conversely, a bee making a correct decision will typically feed and leave the setup without any subsequent scanning of the stimulus features.

These results indicate that the learner group of honeybees tend to be more efficient. They need to scan only one item before making a correct decision (96% of the time), and they need to scan fewer items after making an incorrect choice (1.5 times less than non-learners).

#### DISCUSSION

Our findings confirm the ability of bees to solve the "above and below" visual learning task. The authors of the original study on spatial concept learning in bees (Avarguès-Weber et al., 2011) managed to train all their bees to solve the task, whereas approximately half of our bees failed. The relatively poorer performance of bees in our study may be a result of colony differences or local weather, wind and lighting conditions (Raine and Chittka, 2008; Arnold and Chittka, 2012; Ravi et al., 2016). They might also result from subtle differences in experimental procedures; for example, to facilitate video-tracking, we did not use a lid on the flight arena during experiments, and we took special care to prevent any odor cues or pheromones being deposited on the apparatus by changing the stimuli and washing all tubes before each new trial in the training phase, as well as before tests. In the study by Avarguès-Weber et al. (2011), fresh (unscented) stimuli were used only in tests (not during training), which shows that in their study, bees were able to solve the tasks without the availability of scent, but some of the quantitative differences in learning performance of bees in the two studies might result from the scent cues available during training in Avarguès-Weber et al. (2011).

Individual differences in problem solving abilities are welldocumented in insects (Chittka et al., 2003), especially with difficult tasks (Alem et al., 2016), and it may thus be unsurprising that some individuals failed the task. To explore the question of how the more capable individuals solved the task, it is therefore meaningless to evaluate performance of the entire group, in the same way as one could not study the mnemonic strategies used by people with extraordinary memory capacity by taking a population average that includes all people that lack such capacities. In such cases, one must establish a criterion by which to distinguish the learners from the non-learners. Because of the relatively poor overall performance of bees in our study (compared to that reported by Avarguès-Weber et al., 2011), we chose a relatively lenient criterion (at least 60% overall correct choices during the last 20 trials of learning and at least 70% correct choices during at least one of the two last blocks of 10 trials). Using this criterion, 11 of the 19 bees in our study managed to learn their respective tasks within the 50 training trials and were, on average, able to transfer to the novel stimuli, showing a higher proportion of both first touches and accumulative touches on the stimuli with the correct spatial arrangements (**Figure 2**).

For the question of whether the task was learnt in a manner consistent with concept learning, it is crucial to evaluate whether bees surveyed the arrangement of items in a pair from a distance, and whether the predicted arm of the Y-maze was chosen accordingly. In our study, bees as a group failed to select the

Y-maze arm containing the correct stimuli from the flight arena decision chamber. However, our results for the learner group of bees (that relatively narrowly miss significance at the 5% level) cannot strictly rule out the possibility that, as suggested by Avarguès-Weber et al. (2011), these bees might initiate the decision-making process from a distance, and indeed one individual bee in our study achieved 90% correct choices (from the decision chamber) at the end of training. In our experiments, however, the analysis of the high-speed video footage reveals that much of the decision making process happens when bees were close to the target walls in the Y-maze, when stimuli are scanned close-up, and that the task can be solved without the formal need for concept learning, by simply scanning the bottom item and making decisions accordingly.

The primary aim of this study was to investigate how bees solve the "aboveness" and "belowness" tasks. We aimed to determine what strategies and mechanisms the bees might employ during the learning process, and we therefore video-recorded every single training trial and test. It is generally assumed that the "above and below" task requires a subject to form a conceptual rule to solve the problem, and especially to transfer this ability to novel, correctly configured visual stimuli. However, other explanations might be possible. Three hypotheses were stated in our introduction: bees could recognize the invariant part of each stimulus (the referent), approach it and then depending on whether there is an item (any item) below the referent, decide if it is the correct pattern (simply by noting that the visual field below the referent is empty for "aboveness" learners, or that the visual field above the referent is empty for "belowness" learners). Bees could approach a stimulus scanning only the top (or the bottom) shape and learn that they should either expect the referent in that position, or anything other than the referent (depending on whether they are learning "above" or "below"). Finally, in line with the notion of concept learning, bees could evaluate a whole compound stimulus, using the relative position of the stimuli shapes to determine their "above" or "below" relationship (e.g., scanning both items successively, or viewing the entire arrangement from a distance), and then choose accordingly.

Analysis of the first scanning points showed that the bees were not initially scanning just the referents (crosses), but mostly the lowest shapes presented on the stimuli of the chosen arms (in line with hypothesis 2). This was true for all bees irrespective of their training protocol, or indeed of whether they were successful at the task or not. In approximately two-thirds of cases, the first item scanned by the bees was the lowest presented shape on a stimulus. Furthermore, the second scanning behavior was most often performed in front of a feeder. Therefore, bees did not appear to employ a strategy based on finding the referent (cross in our study) or the spatial relationship between the referent and the other geometrical shape (target). Instead, they used a visual discrimination approach, flying first toward the lower shape and evaluate if it is the referent or not; they do not have to attend to, or indeed learn, anything about the targets. After initially choosing an arm of the Y-maze randomly or according to a side bias, bees trained to the "above" task simply have to decide if the chosen arm contains the referent cross as the lower shape—if yes, they are in the correct arm and can proceed to the feeder. If not, they must have chosen the incorrect arm. Bees trained in the "below" task, finding the referent cross as the lower item, know that they are in the wrong arm; finding "anything but the referent" as the lower item means that they are in the correct arm and can feed. This interpretation is in line with a previous study showing that bees will only learn the lower half of a pattern if this is sufficient to solve a given discrimination task (Giurfa et al., 1999).

# CONCLUSION

Our analysis of the honey bees' flight characteristics showed that the "above and below" problem can be solved using a clever sequential inspection of items rather than, strictly speaking, a spatial concept. By simply flying into a random arm of the Ymaze, or flying into an arm based on a side preference, the task can be solved by inspecting the lower of two shapes in a pair in any arm of the Y-maze, the bee can decide whether it has arrived in the correct arm of the Y-maze or not. It may be tempting to assume that this strategy of solving a seemingly complex learning task might be more suitable for a miniature nervous system such as a bee's, but it will be interesting to explore whether the same strategy may actually be employed by animals with much larger brains when solving similar tasks, such as pigeons (Kirkpatrick-Steger and Wasserman, 1996), chimpanzees (Hopkins and Morris, 1989), baboons (Depy et al., 1999), and capuchins (Spinozzi et al., 2004), or indeed, may be used by humans if they are not verbally instructed how to solve the task. Other studies have reported further forms of concept learning in bees (Giurfa et al., 2001; Avarguès-Weber and Giurfa, 2013; Howard et al., 2018) and in these cases, too, it will be useful to explore the sequential decision making process to see if bees find behavioral strategies to simplify the task or whether concept formation is the most plausible explanation. Finally, it is also possible that bees (and other animals) switch strategies during more prolonged training, so that they might initially learn tasks by close-up inspections of visual targets such as those reported here, and later switch to a more cognitive strategy that allows solving the puzzle from a distance and with higher speed.

Our exploration of the strategy by which bees solve a seemingly complex cognitive task raises questions on the very nature of complexity in comparative cognition. All too often, researchers in that field classify as "advanced cognition" what appears to be clever behavior by casual inspection—but without an analysis of either the behavioral strategy used by animals or a quantification of the computational requirements, or indeed an exploration of the neural networks underpinning the observed behavior (Chittka et al., 2012). Recent computational models of information processing in the bee brain reveal that various forms of "higher order" cognition can emerge as a property of relatively simple neural circuits (Peng and Chittka, 2017; Roper et al., 2017). On the other hand, "simple" associative learning can result in such wide-ranging changes in neural circuitry that these can be detected by sampling just tiny fractions of a principal association region of the bee, the mushroom bodies (Li et al., 2017). These observations, and our analysis of behavior strategies reported here, that the traditional ranking of cognitive operations from simple, non-associative learning through associative learning to apparently more complex such as rule and "abstract concept"

learning may have to be fundamentally revised, and may require more than just asking whether or not animals are clever.

#### ETHICS STATEMENT

This study does not involve human subjects. The 1986 EU Directive 86/609/EEC on the protection of animals used for scientific purposes defines "Animal" as any live non-human vertebrate. Therefore, this does not apply to experimental work on insects.

# AUTHOR CONTRIBUTIONS

MR and LC conceived the study. MG and MR elaborated the project, methods, and collected the data. MG analyzed video

#### REFERENCES


footage. MR participated in the data analysis. MG, MR, and LC wrote the manuscript.

#### ACKNOWLEDGMENTS

We thank Stephan Wolf, HaDi MaBouDi, Clint Perry, and Vera Vasas for helpful comments on the manuscript. This study was supported by HFSP program grant (RGP0022/2014) and EPSRC program grant Brains-on-Board (EP/P006094/1).

#### SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpsyg. 2018.01347/full#supplementary-material


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2018 Guiraud, Roper and Chittka. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Inhibitory Pathways for Processing the Temporal Structure of Sensory Signals in the Insect Brain

Hiroyuki Ai<sup>1</sup> \*, Ajayrama Kumaraswamy<sup>2</sup> , Tsunehiko Kohashi<sup>3</sup> , Hidetoshi Ikeno<sup>4</sup> and Thomas Wachtler<sup>2</sup>

<sup>1</sup> Department of Earth System Science, Fukuoka University, Fukuoka, Japan, <sup>2</sup> Department of Biology II, Ludwig-Maximilians-Universität München, Martinsried, Germany, <sup>3</sup> Neuroscience Institute, Division of Biological Science, Graduate School of Science, Nagoya University, Nagoya, Japan, <sup>4</sup> School of Human Science and Environment, University of Hyogo, Himeji, Japan

Insects have acquired excellent sensory information processing abilities in the process of evolution. In addition, insects have developed communication schemes based on the temporal patterns of specific sensory signals. For instance, male moths approach a female by detecting the spatiotemporal pattern of a pheromone plume released by the female. Male crickets attract a conspecific female as a mating partner using calling songs with species-specific temporal patterns. The dance communication of honeybees relies on a unique temporal pattern of vibration caused by wingbeats during the dance. Underlying these behaviors, neural circuits involving inhibitory connections play a critical common role in processing the exact timing of the signals in the primary sensory centers of the brain. Here, we discuss common mechanisms for processing the temporal patterns of sensory signals in the insect brain.

Keywords: cricket, disinhibition, duration coding, honeybee, moth, postinhibitory rebound, temporal structure, waggle dance

# INTRODUCTION

The temporal patterns of sensory signals can serve as critical cues in behavioral choice. Insects offer a striking advantage over vertebrates for analyzing morphology and physiology of neural circuits at the levels of single identified neurons (Menzel, 2006), so that the network architecture underlying cognitive function can be investigated in detail. Recently, the neurophysiological mechanisms for processing the temporal structure of sensory signals have been revealed in different sensory modalities in various insect species. These results not only suggest that the timed interaction of excitation and inhibition plays key roles in temporal pattern recognition in insects, but also unveil network architectures underlying the coding of sensory temporal structure. Here, we review temporal cognition and its underlying neural mechanisms in the olfactory system of moths and in the auditory systems of crickets and honeybees.

# MOTH ODOR PLUME FOLLOWING

Many insects orient themselves toward conspecifics or food sources using odor cues. As a model system of this behavior, the sex pheromone response of the male moth has been studied extensively. The male moth is attracted by sex pheromones and approaches the odor source, the female.

#### Edited by:

Martin Giurfa, UMR5169 Centre de Recherches sur la Cognition Animale (CRCA), France

#### Reviewed by:

Axel Brockmann, National Centre for Biological Sciences, India Midori Sakura, Kobe University, Japan Berthold Gerhard Hedwig, University of Cambridge, United Kingdom

> \*Correspondence: Hiroyuki Ai ai@fukuoka-u.ac.jp

#### Specialty section:

This article was submitted to Comparative Psychology, a section of the journal Frontiers in Psychology

Received: 01 June 2018 Accepted: 31 July 2018 Published: 21 August 2018

#### Citation:

Ai H, Kumaraswamy A, Kohashi T, Ikeno H and Wachtler T (2018) Inhibitory Pathways for Processing the Temporal Structure of Sensory Signals in the Insect Brain. Front. Psychol. 9:1517. doi: 10.3389/fpsyg.2018.01517

**182**

While approaching the source, the animal changes its flight course if it detects the odor plume (**Figure 1A**; Kaissling, 1997). In turbulent air flow, the odor plume forms a cloud of unevenly distributed odor filaments entangled with nonodorized air pockets (**Figure 1B**; Celani et al., 2014). Therefore, the sensory organs that detect the odor molecules, the antennae, experience intermittent odor pulses. Consequently, as the animal approaches the odor source, it receives the odor pulses in short intervals. Such intermittent detection of sex pheromones is indeed essential for sustained upwind flight (**Figure 1A**; Baker et al., 1985) or walking (Kanzaki, 1997) toward a pheromone source. Flying moths cease to make upwind progress and begin to "cast" across the wind line when they lose the pheromonal stimulus, that is, when the intervals between odor pulses become longer than those in the odor cloud (Baker et al., 1985). The neural circuit for encoding the temporal structure of odor stimuli has been clarified (Christensen et al., 1993, 1998). In the primary olfactory center of the moth antennal lobe, most local interneurons (LNs) use an inhibitory neurotransmitter, gammaaminobutyric acid (GABA), and two types of LNs (named LN1 and LN2 in this review) and projection neurons (PNs) are involved in olfactory signal processing (see **Figure 2A** for a circuit diagram). Indeed, current-induced firing in the LN1 led to hyperpolarization and suppressed firing in the PN (**Figure 1C**). The time course of the PN suppression closely followed the period of current injection in the LN1, and spiking in the PN resumed immediately upon repolarization of the LN1 (**Figure 1C**). Conversely, hyperpolarizing current injected into another LN1 caused an abrupt suppression of firing of the cell, and this resulted in depolarization and firing in the PN (**Figure 1D**). This firing in the PN occurred only during LN1 hyperpolarization. This relationship between LN1 and PN also occurred during sensory stimulation. When the antenna was exposed to the principal sexpheromone component, Bombykal, the LN1 was inhibited, and firing was suppressed (**Figure 1E**). The decreased firing in the LN1 was associated with increased firing in the PN. The period of elevated PN activity approximated the duration of the odor pulses. These results suggest that the firing of the PN is allowed by a disinhibition (inhibition of an inhibitory neuron) of PN1, and that suppression of the inhibitory LN1 is the mechanism underlying the disinhibition (**Figures 2A,B**; Christensen et al., 1993).

Another type of LN, named LN2 here, receives excitatory input from olfactory sensory afferents (Christensen et al., 1998). Since LN2s are inhibitory neurons, their postsynaptic neurons, PNs, show fast inhibitory postsynaptic potentials (IPSPs; I<sup>1</sup> in **Figure 1F**) in response to stimulation of the ipsilateral antenna. IPSP responses of PNs disappeared when the GABA receptor blocker bicuculline was applied, resulting in increased variability in the timing of evoked spikes (**Figure 1F**). When short intermittent pulses of female sex pheromone were applied to the antenna, as in the odor plume, the moth advanced directly toward the odor source, and each stimulus pulse evoked a train of spikes in the PN that was linked to the intermittent stimulus pattern (**Figure 2B**; Christensen et al., 1998).

Thus, the moth uses inhibitory pathways for detecting the timing of both the onset, through LN2, and the continuation, through LN1, of odor pulses (**Figures 2A,B**). A simulation study of the neuronal circuit that processes the temporal structure of olfactory stimuli in moths also indicated that individual PNs, but not individual olfactory receptor neurons (ORNs), encode the onset and offset of odor puffs for any temporal structure of stimuli (Jacob et al., 2017).

# CRICKET CALLING SONGS

Temporal signal processing mechanisms are used not only in olfaction but also in audition. Male crickets produce sound pulses by rubbing their forewings together (**Figure 1G**). Each pulse has a carrier frequency around 5 kHz, and repetitive pulses constitute a chirp (**Figure 1H**; Huber et al., 1989). The temporal structure of male songs is species-specific and is used to attract conspecific females for successful mating. The interpulse interval (IPI) is one of the key parameters underlying this behavior (Hedwig, 2006). IPIs are rather fixed, which is favorable for experimental manipulation and analysis. A recent study by Schöneich et al. (2015) clarified the mechanisms underlying such IPI-selective responses in the Mediterranean field cricket (Gryllus bimaculatus). A chirp of this species consists of 3–4 repetitive sound pulses, with a pulse duration (PD) of 15–23 ms and an IPI of 16–24 ms (**Figure 1H**). Results of electrophysiological experiments suggest the following neural pathway for IPI selection (**Figure 1I**; Schöneich et al., 2015): The male song is received by the tympanic auditory organ, located on the forelegs in females. The sensory afferents of the organ project to an auditory neuropil in the prothoracic ganglion, in which an ascending neuron 1 (AN1) has its dendritic arbor (Wohlers and Huber, 1982). AN1 encodes temporal patterns of the song in its spike trains. While AN1 has a direct excitatory synapse on an excitatory local interneuron 3 (LN3) in the brain, AN1 also indirectly excites LN3 via an inhibitory LN2 and an excitatory LN5. LN2 is directly activated by AN1 during a sound pulse and evokes a lasting hyperpolarization, i.e., inhibition, in its postsynaptic LN5. This inhibition gives rise to a rebound depolarization of the membrane potential (**Figure 1J**, asterisks), a so-called postinhibitory rebound (PIR). LN5 is a graded-potential neuron that does not exhibit action potentials but will release neurotransmitters depending on membrane depolarization; LN5 thus excites its postsynaptic LN3 upon PIR (**Figure 1J**, blue arrows). Because of this circuit organization, LN3 is excited most strongly by pulses with an appropriate IPI in which the delayed excitation from the PIR in LN5 upon a pulse arrives coincidentally with the direct excitation from AN1 evoked by the subsequent pulse (**Figure 1J**, right). Thus, LN3 functions as a coincidence detector that shows selective response to certain IPIs. These results suggest that crickets use a combination of a coincidence detector function and a PIR-based delay mechanism for detecting the timing of pulse patterns in auditory communication signals (**Figures 2C,D**).

Notably, the PIR occurs at a fixed delay of about 40 ms from the end of each pulse, and this corresponds to the pulse period (PP), the sum of PD and IPI. This observation accounts well for the preference of the LN3 response to the overall pulse structure

FIGURE 1 | Insect communication signals and physiologies of critical interneurons involved in temporal processing of sensory signals. (A) Simulated flight path of a moth (winding curve) in a pheromone plume (area enclosed by solid straight lines). Repetitive exposure to sex pheromones in the plume is necessary for sustained upwind flight (from left to right) toward a pheromone source. When the animals lose the pheromonal stimulus, they cease to make upwind progress and instead begin casting. (B) The structure of a plume in turbulent flow from right to left. The shaded area represents the projection of the conical average plume. (C–F) Responses of key neurons in the moth odor processing circuit (see Figure 2A for a circuit diagram). (C) Depolarization-induced firing in the local interneuron 1 (LN1, top) led to hyperpolarization and suppression of firing in the projection neuron (PN, bottom). The PN suppression closely followed the onset of current injection in the LN1 (arrow), and spiking in the PN resumed immediately upon repolarization of the LN1. (D) Hyperpolarizing current injected into an LN1 caused an abrupt suppression of firing of the LN1, and this resulted in depolarization and firing in the PN. This firing in the PN occurred only during LN1 hyperpolarization. (E) Intracellular records from the LN1 and the PN during sex pheromone stimulation (between the two dashed lines). (F) Intracellular records from a local interneuron 2 (LN2, top) and a PN (bottom) responding to brief electrical stimulation of the ipsilateral antennal nerve (asterisks, left). PNs show fast inhibitory postsynaptic potentials (IPSPs; I<sup>1</sup> shown left of PN), which disappeared when the gamma-aminobutyric acid (GABA)-receptor blocker bicuculline was applied, resulting in increased variability in the timing of evoked spikes (right of PN). (G) Male crickets produce a calling song by rubbing both forewings together. (H) Audio signal of the calling song in the Mediterranean field cricket (Gryllus bimaculatus). Females are selectively attracted to the pulse pattern of the conspecific calling song. Each chirp has a temporal structure with a fixed pulse period (PP), consisting of pulse duration (PD) and interpulse interval (IPI). (I) Neural network for detecting the temporal (Continued)

#### FIGURE 1 | Continued

fpsyg-09-01517 August 20, 2018 Time: 16:47 # 4

structure of the male cricket calling song. (J) Intracellular membrane potential records of critical interneurons in cricket auditory processing. A postinhibitory rebound (PIR, indicated by asterisks) excitation plays a critical role in song detection. (K) Moving trajectory of a honeybee during the waggle dance. The dance consists of a waggle phase (WP) and a return phase. The distance to the flower source is encoded as the duration of the WP of the dance. (L) Thoracic vibration velocities recorded during the WP. Intermittent vibration pulses occur with a constant PD of about 16 ms and a PP of about 33 ms. (M) Intracellular records of dorsal lobe interneurons 1 (DL-Int-1, middle) and 2 (DL-Int-2, top) in the primary auditory center of the honeybee (see Figure 2E for a circuit diagram) in response to vibratory mechanical stimulation to an antenna (bottom). Left: When the PPs are shorter than 50 ms, the DL-Int-1 receives strong inhibition that allows no spikes during the pulse trains and exhibits a PIR excitation (arrowheads) upon the offset of the pulse train. DL-Int-2 exhibits elevated spiking activity during stimulation. Right: DL-Int-1 shows spikes (asterisks) intermittently during the IPI phase when the PP of the stimulus is longer than 50 ms. Under this condition, the DL-Int-2 often shows a lack of spikes with remarkable IPSPs (dots). Modified from Kaissling (1997) for A; Celani et al. (2014) for B; Christensen et al. (1993) for C, D, and E; Christensen et al. (1998) for F; Hedwig (2016) for H; Schöneich et al. (2015) for I and J; Hrncir et al. (2011) for K and L, with the permission of Birkhäuser Verlag for A; Springer Nature for C, D, and E; The Society for Neuroscience for F; The American Association for the Advancement of Science (AAAS) for I and J; and The Company of Biologists for K and L.

of conspecific songs; LN3 also responds selectively to PP in addition to IPI (Kostarakos and Hedwig, 2012). Finally, LN4 is also suggested to receive inhibitory inputs from LN2 (**Figure 1I**). Therefore, upon the first stimulus pulse, LN4 cannot evoke spikes even if excitatory input is received from LN3 (**Figure 1J**, left). However, upon the second pulse, LN4 receives a stronger excitation from LN3 due to the PIR-based excitation from LN5, resulting in an overshoot evoking spikes (**Figure 1J**, right). Thus, LN4 can function as a temporal feature detector that shows even sharper selectivity for the combination of conspecific IPI and PP.

#### HONEYBEE WAGGLE DANCE

Honeybees convey the spatial information of profitable flower sources to hive mates using their waggle dance, in which the duration of the waggle phase (WP) increases proportionally with the distance to the flower source (von Frisch, 1967). During the WP, the dancers vigorously shake their abdomens while beating their wings at about 265 Hz (**Figure 1K**). The other individuals follow the dancer's abdomen, receiving intermittent vibration pulses caused by the dancer's wing beats. The pulses have a constant PD of around 16 ms and a PP of around 33 ms (**Figure 1L**). The airborne vibrations are detected by the vibration-sensitive sensory organs in the antennae, called Johnston's organs (JO; Dreller and Kirchner, 1993), and the vibration signals are transmitted to the primary auditory center including the dorsal lobe (DL; Ai et al., 2007; Brockmann and Robinson, 2007). Anatomical and physiological evidence suggests a neural circuit for processing vibration pulses in the honeybee brain (Ai et al., 2017; **Figures 1M**, **2E**). An identified DL neuron, DL-Int-1, is a GABAergic inhibitory neuron (Ai et al., 2017). DL-Int-1 shows spontaneous activity, but when trains of pulses with short pulse period (short PP) are applied to the antenna, DL-Int-1 shows remarkable hyperpolarization and the spontaneous spikes disappear (**Figure 1M**, left column).

The mechanism of this inhibition of DL-Int-1 is still unknown. A PIR excitation (**Figure 1M**, arrowheads) appeared upon the offset of the pulse train. Under this stimulus condition, DL-Int-2, a presumed postsynaptic neuron of DL-Int-1, evokes continuous spikes (**Figure 1M**, top left). In contrast, when trains of pulses with long-PP are applied to the antenna, DL-Int-1 shows intermittent spikes during the train of pulses (**Figure 1M**, right column, asterisks), and DL-Int-2 often shows a lack of spikes with remarkable IPSPs (**Figure 1M**, dots). A computational analysis based on these data suggests that the honeybee may use a disinhibitory network to encode the duration of the WP: DL-Int-2 spiking upon excitatory input from JO afferents is elicited by an inhibition of the presynaptic inhibitory neuron DL-Int-1 (**Figures 2E,F**; Kumaraswamy et al., 2017). Importantly, DL-Int-2 spikes in response to stimulation by trains of pulses with short PP (**Figure 1M**, left), presumably as a result of the short-PP selectivity of the inhibition from DL-Int-1. Thus, the disinhibitory network contributes to the coding of not just the WP, but also the short PP. These experimental and computational results suggest the following motif that resembles the functions of a stopwatch: When a train of vibration pulse stimuli is applied to the JO, DL-Int-1 stops the spontaneous spikes via hyperpolarization. This termination of spontaneous spikes of DL-Int-1 could lead to the timing of spike burst onsets in DL-Int-2, like the start signal in a stopwatch. During the train of vibration pulse stimuli, tonic hyperpolarization of DL-Int-1 could sustain the spike burst on DL-Int-2 as long as the PPs are within the appropriate range, corresponding to a running stopwatch. When the vibration pulses stop, DL-Int-1 shows a PIR excitation, which inhibits the spike burst of DL-Int-2, like the stop signal in a stopwatch. Interestingly, sustained inhibition of a critical auditory neuron plays an important role in the selectivity for stimulus duration not only in insects, but also in anurans (Alluri et al., 2016). These findings suggest a common function across sustained inhibition in various species.

In addition to the airborne vibration caused by wingbeats, tactile contact of a follower's antenna with the dancer's body may also function as a signal related to the WP (Dyer, 2002; Michelsen, 2003; Gil and De Marco, 2010). The tactile contact deflects the antenna, which may be detected by neurons in the antennal joint hair sensilla. These sensory afferents also project to the DL (Ai et al., 2007), implying that the identified DL interneurons discussed here might also be involved in the processing of the temporal structure of the tactile contacts.

#### DISCUSSION

In this review, we compared the processing of the temporal structure of sensory signals in different modalities and different insect species. We particularly focused on the roles of inhibitory interneurons in determining the spike timing of postsynaptic neurons and thereby contributing to extracting temporal features. We highlighted two common characteristics, disinhibition and PIR, found across temporal processing circuits.

In the olfactory processing in moths and in the vibration processing of waggle dance signals in honeybees, a disinhibition is suggested to contribute to detecting the total duration of sensory stimulation (**Figures 2B,F**). In the anuran auditory system, a disinhibitory circuit motif was also proposed for counting the number of sound pulses that occur with a speciesspecific IPI (Naud et al., 2015). Thus, disinhibition might be a common mechanism across the animal kingdom for encoding the total duration of sensory input, which is equivalent to the product of the number of pulses with a fixed PP. PIR can serve to process IPIs in the cricket song (**Figure 2D**) and the offset of the WP in honeybees (**Figure 2F**). PIR also occurs in the mammalian auditory system, underlying selectivity for periodic low frequency amplitude modulations of sound signals (Felix et al., 2011) and, at a different time scale, spatial selectivity for sound location (Beiderbeck et al., 2018). Among the three insect models we reviewed here, the temporal processing circuits of the honeybee waggle dance employ both disinhibition and PIR (**Figure 2F**), allowing reliable encoding of WP; disinhibition results in a sustained response of the output neuron, DL-Int-2, to continuously detect waggling, and PIR excitation results in a signaling stimulus offset in the same neuron.

Inhibitory interneurons in insects have been suggested to be involved in various other aspects of sensory processing such as gain control (Olsen and Wilson, 2008), lateral inhibition (Sachse and Galizia, 2002; Silbering and Galizia, 2007), synchronization of spikes (MacLeod and Laurent, 1996; Bazhenov et al., 2001), and encoding of temporal stimulus patterns (Christensen et al., 1993, 1998; Hedwig, 2006, 2016). Here, we also explored the similarities between the neural processing of honeybee waggle dance signals and the neural processing of cricket audition and moth olfaction. Inhibitory inputs are also suggested to be critical in temporal processing in vertebrates. Future experiments should further elucidate the role of postsynaptic inhibition in encoding temporal signaling and in decoding the distance to a flower source in the foraging flight of the honeybee.

#### AUTHOR CONTRIBUTIONS

All authors listed have made a substantial, direct and intellectual contribution to the work, and approved it for publication.

## FUNDING

This research was supported by Grants-in-Aid for Scientific Research from the Ministry of Education, Science, Technology, Sports, and Culture of Japan (Grant No. 18K06345), and a grant from the Central Research Institute of Fukuoka University (Grant No. 171031), and a grant from the German Federal Ministry of Education and Research (BMBF, Grant No. 01GQ1116).

# ACKNOWLEDGMENTS

We would like to thank Dr. Midori Sakura for the photo of the stridulating cricket in **Figure 1G**. We are grateful to the reviewers for valuable suggestions.

## REFERENCES

fpsyg-09-01517 August 20, 2018 Time: 16:47 # 6


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2018 Ai, Kumaraswamy, Kohashi, Ikeno and Wachtler. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Associative Learning of Stimuli Paired and Unpaired With Reinforcement: Evaluating Evidence From Maggots, Flies, Bees, and Rats

Michael Schleyer<sup>1</sup> \*, Markus Fendt2,3, Sarah Schuller<sup>1</sup> and Bertram Gerber1,3,4

<sup>1</sup> Department Genetics of Learning and Memory, Leibniz Institute for Neurobiology, Magdeburg, Germany, <sup>2</sup> Institute for Pharmacology and Toxicology, Otto von Guericke University Magdeburg, Magdeburg, Germany, <sup>3</sup> Center for Behavioral Brain Sciences, Magdeburg, Germany, <sup>4</sup> Behavior Genetics, Institute for Biology, Otto von Guericke University Magdeburg, Magdeburg, Germany

#### Edited by:

Jeffrey A. Riffell, University of Washington, United States

#### Reviewed by:

Kosuke Sawa, Senshu University, Japan Lucia Regolin, Università degli Studi di Padova, Italy

\*Correspondence: Michael Schleyer michael.schleyer@lin-magdeburg.de

#### Specialty section:

This article was submitted to Comparative Psychology, a section of the journal Frontiers in Psychology

Received: 23 March 2018 Accepted: 30 July 2018 Published: 24 August 2018

#### Citation:

Schleyer M, Fendt M, Schuller S and Gerber B (2018) Associative Learning of Stimuli Paired and Unpaired With Reinforcement: Evaluating Evidence From Maggots, Flies, Bees, and Rats. Front. Psychol. 9:1494. doi: 10.3389/fpsyg.2018.01494 Finding rewards and avoiding punishments are powerful goals of behavior. To maximize reward and minimize punishment, it is beneficial to learn about the stimuli that predict their occurrence, and decades of research have provided insight into the brain processes underlying such associative reinforcement learning. In addition, it is well known in experimental psychology, yet often unacknowledged in neighboring scientific disciplines, that subjects also learn about the stimuli that predict the absence of reinforcement. Here we evaluate evidence for both these learning processes. We focus on two study cases that both provide a baseline level of behavior against which the effects of associative learning can be assessed. Firstly, we report pertinent evidence from Drosophila larvae. A re-analysis of the literature reveals that through paired presentations of an odor A and a sugar reward (A+) the animals learn that the reward can be found where the odor is, and therefore show an above-baseline preference for the odor. In contrast, through unpaired training (A/+) the animals learn that the reward can be found precisely where the odor is not, and accordingly these larvae show a below-baseline preference for it (the same is the case, with inverted signs, for learning through taste punishment). In addition, we present previously unpublished data demonstrating that also during a two-odor, differential conditioning protocol (A+/B) both these learning processes take place in larvae, i.e., learning about both the rewarded stimulus A and the non-rewarded stimulus B (again, this is likewise the case for differential conditioning with taste punishment). Secondly, after briefly discussing published evidence from adult Drosophila, honeybees, and rats, we report an unpublished data set showing that relative to baseline behavior after truly random presentations of a visual stimulus A and punishment, rats exhibit memories of opposite valence upon paired and unpaired training. Collectively, the evidence conforms to classical findings in experimental psychology and suggests that across species animals associatively learn both through paired and through unpaired presentations of stimuli with reinforcement – with opposite valence. While the brain mechanisms of unpaired learning for the most part still need to be uncovered, the immediate implication is that using unpaired procedures as a mnemonically neutral control for associative reinforcement learning may be leading analyses astray.

Keywords: safety learning, fear conditioning, reward, punishment, memory valence

# INTRODUCTION

fpsyg-09-01494 August 24, 2018 Time: 12:7 # 2

Finding rewards and avoiding punishments are powerful goals of behavior in insects and vertebrates, including humans. To maximize rewards, for example, it is beneficial to learn about the stimuli that predict where and when they can be found. However, it can be equally important to learn where and when a reward will not be found (Rescorla, 1967; Malaka, 1999). Although well established in the classical experimental psychology literature, the latter learning process is frequently left out of consideration even in immediately neighboring fields of study. This can be problematic because research into the brain mechanisms of learning and memory, for example, may go astray if it fails to take into account both of these processes when designing control procedures for the effects of associative learning. Here we focus non-exclusively on two cases of Pavlovian conditioning, one in larval Drosophila and the other in rats, which provide different types of control procedure for determining a baseline behavior against which the effects of learning both through reinforcement and non-reinforcement can be assessed.

In Pavlovian conditioning, a stimulus A (in Pavlovian terminology: the conditioned stimulus or CS) is presented along with a reinforcer + (in Pavlovian terminology: the unconditioned stimulus or US). By such paired A+ training, an association is formed between A and the reinforcer (Pavlov, 1927). In the past few decades, powerful theories have been introduced to explain such reinforcement learning. Many of them feature what is known as the delta rule (Rescorla and Wagner, 1972; Mackintosh, 1975; Rumelhart et al., 1986; Van Hamme and Wasserman, 1994; Malaka, 1999) (**Supplementary Figure S1A**). Essentially, this rule holds that the more we remember, the less we learn. In other words, the amount of reward learning about A depends on the difference between the reward received in the presence of A minus the reward predicted by A, the so-called 'prediction error'. Considering multiple training trials (A+, A+, A+, etc.), the prediction error is large and positive for the first A+ trial. This is because much more reward is received than is predicted ('pleasant surprise'). As training progresses, the reward will eventually be fully predicted such that the prediction error is zero and no further learning accrues to A.

Already in early studies in the field, unpaired training was introduced as a control procedure for reinforcement learning (e.g., Harris, 1943). In such a procedure, A and the reinforcer never occur in temporal proximity (A/+ for the case of reward learning, A/− for the case of punishment learning). Later, however, Rescorla (1966, 1967, 1968) demonstrated that animals can learn through such unpaired presentations: specifically, they can learn that A predicts the absence of the reinforcer. How is this possible? Doesn't it violate the principles of association to suggest that a stimulus A presented without reinforcement is learned about? And if A is presented unpaired from reward, for example, how is it possible that A comes to predict where reward is not, rather than where punishment, or the spaghetti monster, is not? In fact, delta-rule types of model for reinforcement learning can offer an explanation. The assumption is that during for example a reward-only trial (+), an association is formed between the experimental context and the reward (Dweck and Wagner, 1970; Rescorla, 1972; Grau and Rescorla, 1984; Bouton and Nelson, 1998) ('context' being understood as the totality of stimuli that are not manipulated during the experiment). When in a subsequent trial A is presented within the same context, this context-reward association will predict the reward. As the reward is not actually present, however, a negative prediction error arises: less reward is received than is contextually predicted ('unpleasant surprise'). This negative prediction error will then be associated with A, which in consequence becomes a signal for no-reward (rather than remaining neutral, i.e., not being a signal for anything) (Rescorla, 1966, 1967, 1968). Thus, as a result of A+ training, A signals where the reward can be found, whereas after A/+ training A signals where the reward cannot be found. Most of the remainder of the present paper is about strategies for studying A+ and A/+ learning, and about the implications of these learning processes for designing control procedures for reinforcement learning. We first review the literature on larval Drosophila that provides evidence for A+ and A/+ learning relative to a control condition that prevents the behavioral expression of associative memories. Then we report on so far unpublished experiments regarding these learning processes in differential conditioning in this paradigm, and briefly evaluate pertinent literature on adult flies, honey bees and rats. Finally, we present unpublished data demonstrating associative learning through paired and unpaired training in a fear-conditioning paradigm in rats, relative to a control condition that prevents the formation of associative memories.

# UNPAIRED-MEMORY IN LARVAL DROSOPHILA?

Odor-taste associative learning in the Drosophila larva is an ecologically plausible study case for Pavlovian conditioning (reviews include Gerber and Stocker, 2007; Diegelmann et al., 2013; Widmann et al., 2017; see also Aceves-Pina and Quinn, 1979 for a pioneering approach using odor-electric shock learning). In Pavlovian terminology, the odor would be designated the CS, and the tastant the US. The rich toolbox for transgenic manipulation available for Drosophila, the numerically simple larval brain consisting of only about 10,000 neurons,

and the upcoming cellular atlas and synaptic connectome of its nervous system allow for experiments with enticing analytical resolution (Venken et al., 2011; Li et al., 2014; Eichler et al., 2017). Despite the simplicity of their brains, larvae learn to associate odor stimuli with taste rewards such as sugar, or with bitter tastants such as quinine as a punishment (Scherer et al., 2003; Gerber and Hendel, 2006). They further show discrimination, generalization, memory consolidation, and an organization of learned behavior according to its expected outcome (Gerber and Hendel, 2006; Mishra et al., 2010; Chen et al., 2011; Schleyer et al., 2011; Chen and Gerber, 2014; Schleyer et al., 2015a,b; Widmann et al., 2016). Last but not least, the transparent cuticle of larvae allowed for the first use of Channelrhodopsin-2 to remote-control central brain neurons in a behaving animal (Schroll et al., 2006). Thus, the larva is simple enough to be studied with ease and precision, and complex enough for this to be worth the effort.

# How to Determine Baseline Odor Preferences in Larval Drosophila

For both larval and adult Drosophila, one-odor 'absolute' conditioning paradigms are available (larvae: Saumweber et al., 2011a; adults: Niewalda et al., 2011). For example, larvae are repeatedly transferred between two types of Petri dish featuring substrates that are supplemented, or not, with a taste reward (**Figure 1**). An odor A is presented together with a sugar-containing substrate; a tasteless substrate is then presented without an odor (A+/blank, paired training). These animals can learn that the reward can be found where the odor is (see also **Supplementary Figure S1B**). Importantly, a second group of larvae is trained unpaired, i.e., the odor and the reward are present on different dishes (A/+, unpaired training). These animals can learn that the reward can be found where the odor is not (see also **Supplementary Figure S1C**). After typically three such training cycles, the animals are transferred to a test Petri dish where their preference for A is assessed. This usually reveals a higher preference for A after paired than after unpaired training (**Figure 1A**, the two left-most box-plots of each panel). This difference in preference between pairedtrained and unpaired-trained animals indicates how much the contingency between odor and reward matters for the larvae's odor preference, and can thus serve as a measure of associative memory. However, is this due to associative memory in the paired-trained group, associative memory in the unpairedtrained group, or both? The observation that the larvae approach or avoid the odor after a given training procedure is not in itself an argument in this respect, because odors are not neutral to experimentally naive larvae, but support moderate levels of attraction (**Figure 2**) (Cobb, 1999; Saumweber et al., 2011a). This being so, can the behavior of experimentally naive larvae be used as a baseline against which to measure effects of paired and unpaired training? We argue that such a comparison would be misguided. Relative to both pairedand unpaired-trained animals, experimentally naive animals lack not only the target associative experiences, but also the experience of handling, of exposure to the odor, and of exposure to the reward – experiences that can evidently all affect odor preference (larvae: Boyle and Cobb, 2005; Michels et al., 2005; Colomb et al., 2007; Saumweber et al., 2011b; adults: Préat, 1998; Sadanandappa et al., 2013; Niewalda et al., 2015; Hattori et al., 2017). The same applies to measures of odor preference after handling-only (lacking the target associative experience and exposure to the odor and the reward), after odor-only exposure (lacking the associative experience and reward-exposure), or after reward-only exposure (lacking the associative experience and odor-exposure). In other words, using any of the abovementioned procedures to establish a baseline odor preference can lead analyses of associative memory astray. A better option would be to expose animals to both odor and reward with a truly randomized temporal relationship between them (Rescorla, 1966, 1967, 1968). In such a randomized procedure, the probability of the reinforcer occurring would be the same in the presence as in the absence of the odor, and the odor would thus not provide any information about the reinforcer. It has been shown that animals may nevertheless associatively learn in such a procedure, depending on the specific parameters of the experiment and the exact sequence of events (discussed in Rescorla, 1972; Papini and Bitterman, 1990). Still, if that appropriate parameters are used, the truly randomized procedure can provide a baseline against which to measure the effects of paired and unpaired training. Indeed, it has been successfully used in the case of fear conditioning in the rat, for example, as will be discussed in the Section "Unpaired-Memory in Rodents?". Even so, a randomized procedure is only feasible if training consists of sufficiently many trials (Rescorla, 1972), and not in cases when only a handful of trials are used, as in the paradigms discussed for Drosophila and honeybees.

A second strategy is not to try to prevent the formation of associative memory, but to prevent its behavioral expression. How can this be done? Fortunately, the behavioral expression of associative memory in larvae has been found to depend on the circumstances of testing (Hendel et al., 2005; Schleyer et al., 2011, 2015a,b; Paisios et al., 2017). Specifically, the behavioral expression of odor-reward memories both after paired and after unpaired training is fully suppressed when the test is carried out in the presence of the reward. That is, if the test is conducted in the presence of the reward, the larvae behave the same toward the odor regardless of whether they have undergone paired or unpaired training. This can be understood as adaptive if one views conditioned behavior as a search for reward that is obsolete if the soughtfor reward is already present (Gerber and Hendel, 2006; Schleyer et al., 2011, 2015a,b; see also Craig, 1918). Significantly, animals tested in this way have experienced the same amount of handling, odor exposure and sugar exposure, and will have even formed the same associative memories as animals trained the same but tested in the absence of the reward. Thus, none of these aspects of experience can account for differences in test behavior in the presence versus in the absence of the reward. What the presence of the reward during the test does is to prevent the behavioral expression of associative memory, i.e., to abolish the difference in odor preference between paired-trained and unpaired-trained animals (**Figure 1**, green box plots). This is an effect specific to

FIGURE 1 | Paired and unpaired memory in larval Drosophila. (A) In five independent experiments from four different previously published studies, paired and unpaired memory was demonstrated using n-amyl acetate as the odor (AM, red cloud) and fructose as the reward (green-filled circles). In a Petri-dish assay, odor and reward were presented paired such that the odor was presented while the animals were on a reward-containing Petri dish; the animals were then transferred to an empty Petri dish without odor or reward (white-filled circle). In an independent group of animals, odor and reward were presented unpaired, in consecutive trials. When tested for their preference for the odor (AM Pref), the larvae preferred the odor more after paired than after unpaired training (white-filled box plots). When tested on a reward-containing Petri dish, however, the larvae displayed an intermediate level of odor preference that was the same regardless of the training regimen (green-filled box plots). This can therefore serve as a baseline for odor preference in animals that have established, but do not behaviorally express, associative odor memory (stippled line). Such a procedure reveals that memory through reward-paired training increases, whereas memory through reward-unpaired training decreases odor preference relative to this baseline. (B) Same as (A), but using either 1-octanol (1-OCT, blue cloud) or 3-octanol (3-OCT, purple cloud) as odors.

#### FIGURE 1 | Continued

fpsyg-09-01494 August 24, 2018 Time: 12:7 # 5

(C) In two independent, previously published experiments, paired and unpaired memory was demonstrated using n-amyl acetate as the odor (AM, red cloud) and quinine as punishment (yellow-filled circles). Only when tested for their odor preference on a punishment-containing Petri dish did the larvae avoid the odor more after paired than after unpaired training (yellow-filled box plots). When tested in the absence of punishment, the larvae displayed an intermediate level of odor preference that was the same regardless of the training regimen (white-filled box plots, yellow stippled line). Thus, punishment-paired training decreases, whereas punishment-unpaired training increases odor preference relative to the baseline odor preference shown in the absence of the punishment. Data were taken from the publications indicated above each experiment. For more details on experimental parameters and the methods used, see Table 1 as well as the Methods sections of the indicated papers. Box plots indicate the median as the middle line and the 25/75% and 10/90% quantiles as box boundaries and whiskers, respectively. Sample sizes are displayed below each box-plot. In all cases, the preference values were statistically indistinguishable between the training regimens when animals were tested under baseline conditions [Mann–Whitney U-tests (MW), P > 0.05 corrected according to Bonferroni–Holm within each experiment], indicated by a common letter and a vertical bar above the box plots. The stippled line indicates the median of the pooled preference data under baseline conditions. The preferences after paired and after unpaired training differed from each other, as well as from baseline (MW, P < 0.05 corrected according to Bonferroni–Holm within each experiment), as indicated by different letters above the box plots. For detailed statistical results see Supplementary Table S1.

learned behavior, as olfactory behavior in experimentally naive larvae is not likewise affected (**Figure 2**). The equal level of odor preference in paired-trained and unpaired-trained animals in the presence of the reward can thus be used as a baseline, reflecting olfactory behavior specifically cleared of associative memories. The following Section "Evidence for Unpaired-Memory in Larval Drosophila" discusses what a reanalysis of previously published experiments using such a baseline approach can reveal about the memories formed though paired and unpaired training of odor and taste reinforcement.

### Evidence for Unpaired-Memory in Larval Drosophila

The first experiment including such a baseline condition was reported by Saumweber et al. (2011a) with n-amyl acetate as the odor and fructose as the reward. In this and the following analyses, we pooled the data for paired-trained and unpaired-trained animals tested under baseline conditions (e.g., **Figure 1A**, green box plots and stippled line), and compared them to animals that were paired-trained or unpaired-trained and tested under non-baseline conditions (e.g., **Figure 1A**, blank box plots), using pairwise statistical tests (for details, see the "Materials and Methods" section in the **Supplementary Presentation S1**). Associative memory after paired or unpaired training would manifest itself as a difference between the respective group and the baseline. Indeed, paired odor-reward training increased odor preference compared to baseline, whereas unpaired training decreased odor preference compared to baseline (**Figure 1A**). This result has been reproduced four times in three follow-up studies (**Figure 1A**) (Schleyer et al., 2011, 2015b; Paisios et al., 2017) and confirmed using two further odors (**Figure 1B**) (Saumweber et al., 2011a). Interestingly, it was shown that the resulting increase and decrease in odor preference, respectively, come about by opposite modulations of the microbehavioral tendencies that underlie chemotaxis. After paired training, larvae turn less while moving toward the odor source, turn more while moving away from it, and bias the direction of their turns more toward the odor source than animals under baseline conditions do; after unpaired training, the larvae modulate the very same parameters of their locomotion, yet in the opposite way (**Figure 3**) (Schleyer et al., 2015b; Paisios et al., 2017). Together these analyses show that Drosophila larvae do indeed acquire associative memories during paired training and during unpaired training, and that these memories are opposite in valence and in the 'sign' of microbehavioral modulation. What about the aversive domain?

Drosophila larvae can be conditioned to associate odors with taste punishment such as highly concentrated salt, or quinine (Gerber and Hendel, 2006; Niewalda et al., 2008; El-Keredy et al., 2012). Importantly, the associative memories established by such training are behaviorally expressed only in the presence but not in the absence of the taste punishment (Hendel et al., 2005; Gerber and Hendel, 2006; Schleyer et al., 2011, 2015a; Paisios et al., 2017). This can be understood if conditioned behavior after punishment training is viewed as an escape from the punishment, which is obsolete in the absence of anything to escape from (Gerber and Hendel, 2006; Schleyer et al., 2011, 2015a; see also Craig, 1918). Given that innate olfactory behavior in experimentally naive animals is not likewise affected by the presence of punishing tastants (**Figure 2**), this makes it possible to measure odor preference after paired or unpaired punishment training, and to compare the levels of preference against baseline – which in this case is determined by testing the animals in the absence of the punishment. It turned out that after paired odor-punishment training, larvae prefer the odor less than at baseline, whereas after unpaired punishment training they prefer the odor more than at baseline (**Figure 1C**) (Schleyer et al., 2011; Paisios et al., 2017). In other words, after paired training the larvae seek to escape from the punishment by heading where the odor is not, whereas after unpaired training they seek to escape from the punishment by heading where the odor is. In terms of microbehavior, the comparison to baseline revealed that turn rate and turn direction are modulated in opposite ways after paired versus unpaired punishment-training (Paisios et al., 2017). Specifically, memories after reward-paired and punishmentunpaired training affect these aspects of locomotion in the same way, whereas opposite modulations were observed after both reward-unpaired and punishment-paired training (**Figure 3**) (Paisios et al., 2017).

These results show two points of conceptual relevance. Firstly, the way in which microbehavior is affected is determined by memory valence, not by the used reinforcer: for example, when heading toward the odor source turns are suppressed both by reward-paired and by punishment-unpaired memory (**Figure 3**). This is adaptive because in both cases it keeps the larvae on target (i.e., on track toward the odor). Secondly,

P > 0.05). Small letters above box plots indicate significant differences between odor preferences (MW, P < 0.05 corrected according to Bonferroni-Holm within each experiment). For detailed statistical results see Supplementary Table S2. Other details as in Figure 1.

the way in which the presence of the reinforcer during the test affects the behavioral expression of memory is determined in turn by the used reinforcer, not by memory valence (**Supplementary Figure S2**): for example, the presence of the reward suppresses the behavioral expression of reward-memory both after reward-paired and after reward-unpaired training, although these two types of training establish memory of opposite valence. This is adaptive because in both cases learned behavior is about obtaining the desired outcome (i.e., the reward).

Unpaired learning may explain otherwise enigmatic observations, for example that mutations of learning-related genes affect odor preference after paired and unpaired training with opposite sign (Michels et al., 2011; Saumweber et al., 2011b; Kleber et al., 2016). Likewise, training with higher concentrations of a reward or higher intensity of punishment has opposite effects on odor preference after paired versus unpaired training (El-Keredy et al., 2012; Schleyer et al., 2015a).

#### Unpaired-Memory After Differential Conditioning of Larval Drosophila

Traditionally, most studies of Pavlovian conditioning in Drosophila employ differential, two-odor conditioning (adults: Quinn et al., 1974; Tempel et al., 1983; Tully and Quinn, 1985; larvae: Aceves-Pina and Quinn, 1979; Scherer et al., 2003; Neuser et al., 2005). These procedures are identical to the conditioning paradigms described above, except that an additional odor B is introduced. The larvae receive one odor paired with reinforcement whereas another odor is presented alone (i.e., unpaired from reinforcement) (A+/B training). Subsequently, they are tested for their choice between A and B. If after such A+/B training the animals prefer A over B, this is usually interpreted as caused by the A+ association. Arguably, however, such preference for A over B may be driven by two associative behavioral tendencies: the animals may be attracted to A because it signals where the reward is, and/or they may be repelled by B because it signals where the reward is not (see also Rescorla, 1969 and references therein). Thus, in this type of paradigm it is impossible to disentangle the contribution of either of these two processes. This is required, however, to fully appreciate how experience with reinforcement shapes behavior.

To address this problem, we modified the differential, twoodor conditioning paradigm (see "Materials and Methods" section in the **Supplementary Presentation S1**; see also Saumweber et al., 2011a; for adults: Barth et al., 2014). In these previously unpublished experiments, we first trained larvae 'normally' such that in one group of animals odor A was paired with a fructose reward but odor B was presented alone (A+/B), whereas in an independent group of animals, contingencies were reversed (A/B+). However, we then did not test the animals for their choice between odor A and B, but rather determined their absolute preference for odor A versus blank. This allowed us to assess the preference for odor A after it had been presented, during differential conditioning, either paired or unpaired with the reward. These preferences for odor A were then compared to baseline, i.e., to the preference for odor A after the same type of training but tested in the presence of the reward. This revealed that the preference for odor A is above baseline if, during differential conditioning, it has been the paired-trained odor, and below baseline if it has been the unpaired-trained odor (**Figure 4A**). We conclude that during differential conditioning, too, larvae learn both about the reward-paired and about the reward-unpaired odor. The same principle, with reversed sign, applies in the aversive domain as well (**Figure 4B**).

# UNPAIRED-MEMORY IN ADULT FLIES AND HONEYBEES?

The 'baseline approach' discussed above has so far only been used in larval Drosophila. Is there evidence from other kinds of experimental approach warranting the conclusion that unpaired learning takes place in adult flies, or honeybees?

#### Adult Flies

To the best of our knowledge, no unequivocal, direct evidence is available from adult Drosophila that unpaired learning takes place. However, for a number of observations unpaired memory is a parsimonious explanation.

Using absolute conditioning paradigms with odor and electric shock as a punishment, preference scores have in some studies been reported separately for paired-trained and unpaired-trained flies. In the study by Niewalda et al. (2011), for example, avoidance was found for four different odors after paired odor-punishment training, whereas unpaired training resulted in odor attraction (see also Yarali et al., 2009; Barth et al., 2014; König et al., 2017). The latter result is suggestive of unpaired memory because the odors in question, and in fact odors in general, are innately repulsive to adult Drosophila in the type of setup used (de Belle and Heisenberg, 1994; Préat, 1998; Acevedo et al., 2007; Knapek et al., 2010; Niewalda et al., 2015). Indeed, when the effect of odor concentration on memory performance was evaluated, increasing the odor concentration increased the odor attraction observed after unpaired training (Niewalda et al., 2011) – whereas in experimentally naive flies increasing the odor concentration makes the odors more aversive (Tully and Quinn, 1985). Still, it remains possible that such odor attraction reflects the effects of handling, or of shock-exposure, or of odor-exposure that are part of the training experience. Because training-like odor-exposure and training-like shock-exposure typically only decrease aversion without converting it into attraction (Préat, 1998; Knapek et al., 2010), however, unpaired-memory seems to be the more likely explanation for these effects.

Similarly suggestive data were recently reported by Cohn et al. (2015) from an isolated-brain preparation. The authors used stimulation of olfactory interneurons of the so-called mushroom body ('odor') paired or unpaired with activation of dopaminergic reward neurons (DANs) innervating them. They then measured the physiological effect of such training at the level of the output neurons of the mushroom body (MBONs) using Ca2+-imaging. 'Olfactory' activation paired with DAN activity depressed subsequent MBON activity in response to 'odor,' whereas upon unpaired presentations MBON activity was potentiated. Arguably, and with the same caveats in mind as discussed in the preceding paragraph, these opposite modulations of MBON activity could reflect paired and unpaired memory.

#### Honeybees

Honeybees are a widely used study case for learning and memory, both under natural and under laboratory conditions (Giurfa, 2007; Menzel, 2012). In the present context, studies using Pavlovian reward learning of the proboscis extension response (PER) are particularly relevant. Individual honeybees are harnessed such that they can freely move their antennae and mouthparts, including their proboscis. When their antennae

FIGURE 4 | Paired and unpaired memory upon differential conditioning. (A) In this previously unpublished experiment, larvae were trained in the differential, two-odor version of the learning experiment. The experiments followed established procedures (Gerber et al., 2013; Michels et al., 2017). In brief, we used 2 mol/L fructose as the reward, and n-amyl acetate diluted 1:20 in paraffin (AM, red cloud) as well as undiluted 1-octanol (1-OCT, blue cloud) as odors. Larvae were trained such that AM and 1-OCT were presented in consecutive trials for 2.5 min each. In this differential conditioning paradigm, for one group of animals AM was always presented on a reward-containing Petri dish and 1-OCT on a tasteless Petri dish (left-most box plot). A second group was trained reciprocally, such that 1-OCT was always paired with reward (second box-plot from the left). After three such training cycles, the larvae were transferred to a test Petri dish, where their odor preference for AM was determined. When tested in the absence of the reward, the larvae prefer AM more after it was paired with the reward than after it was not paired with the reward (white-filled box plots). When tested in presence of the reward, the animals display an intermediate level of odor preference that is the same regardless of the training regimen (green-filled box plots). Thus, when AM was trained paired with the reward during differential conditioning, the larvae learned that it signals where the reward is, whereas when AM was trained unpaired with the reward during differential conditioning, the larvae learned that it signals where the reward is not. (B) As in (A), but using 5 mmol/L quinine hemisulfate as punishment. When tested in the presence of the punishment, the larvae avoid AM more after it was paired with the punishment during differential conditioning than after unpaired training (yellow-filled box plots). When tested in the absence of the punishment, the larvae display an intermediate level of odor preference that is the same regardless of the training regimen (white-filled box plots). Thus, when AM was trained paired with the punishment during differential conditioning, the larvae learned that it signals where the punishment is, whereas when AM was trained unpaired with the punishment during differential conditioning, the larvae learned that it signals where the punishment is not. Sample sizes are displayed below each box plot. For detailed statistical results see Supplementary Table S3. For detailed methods, see the "Materials and Methods" section in the Supplementary Presentation S1. Other details as in Figure 1.

are touched with a sucrose solution as a reward, the bees reflexively extend their proboscis and lick the sucrose; few if any such PERs are typically observed when odors are presented to experimentally naive animals. During PER conditioning, an odor A is presented shortly before a reward (A+; in Pavlovian terminology the CS and US, respectively). After such paired training, increased levels of PER are observed in response to the odor alone. Obviously, without modification this paradigm cannot detect memories of opposite valence after unpaired training (A/+): as spontaneous PER rates are low they remain low after unpaired training. In other words, there is no 'negative' PER that could reveal unpaired-memory. One modification allowing such unpaired-memory to be detected is called retardation of acquisition. In such a two-phase paradigm the bees of two independent experimental groups first receive either paired or unpaired reward-training. In a second training phase, the bees of both groups receive paired training (paired-paired group: A+ training followed by A+ training; unpaired-paired group: A/+ followed by A+). As first reported by Bitterman et al. (1983), during the second training phase the bees in the unpairedpaired group respond less to A than those in the paired-paired group. Given that presentations of odor-alone or of reward-alone during the initial training phase do not have such an effect, this shows that unpaired training establishes an associative memory opposite in valence to paired training in the PER paradigm.

Data from two-odor, differential PER conditioning are consistent with, but are not in themselves conclusive evidence for, unpaired learning. In the course of an extended differential conditioning phase (A+, B, A+, B, A+, B, etc.), levels of PER toward B are initially elevated, arguably because of generalization from the first A+ training trial. As training progresses, however, the response levels to B decrease (Bitterman et al., 1983; Komischke et al., 2002; Boitard et al., 2015; see also Tedjakumala and Giurfa, 2013 for similar results in the aversive domain). This could be due either to a loss of generalized memory (the end-state being no memory for B) or to unpaired learning (the end-state being unpaired-memory for B).

The physiological data regarding the effects of unpaired training in the honeybee are complex. The PE1 neuron, an MBON from the peduncle of the mushroom body, has been shown to decrease its activity to an odor that was previously trained paired with reward (Mauelshagen, 1993; Okada et al., 2007). Regarding an unpaired-trained odor, mild increases or decreases in PE1 activity can be observed depending on trial number and time after odor onset (Mauelshagen, 1993; Okada et al., 2007). Also in other MBONs, in the antennal lobe and in the octopaminergic rewarding VUMmx1 neuron, altered responses to unpaired odors have been observed (Hammer, 1993; Faber et al., 1999; Strube-Bloss et al., 2011). These effects are typically small compared to the effects of reward-paired odors, and in no case have proper baseline levels of activity been determined. Therefore, alternative interpretations of these physiological data, such as non-associative learning or extinction learning, which are well documented in honeybees (Braun and Bicker, 1992; Hammer et al., 1994; Menzel et al., 1999; Eisenhardt and Menzel, 2007; Eisenhardt, 2014), remain tenable.

FIGURE 5 | Paired and unpaired memory in rats. In this previously unpublished experiment replicating a study by Andreatta et al. (2012), independent groups of rats were submitted to paired training of a light stimulus and punishment (left), to unpaired training (middle), or to a truly random procedure (right). Specifically, a light stimulus (red circle) and a mild foot-shock (yellow flash) were presented 15 times. For paired training, the light stimulus immediately preceded the shock (intertrial interval, ITI: 90–150 s); for unpaired training, the light stimulus and the shock were temporally separated from each other by at least 12 s; and in the random procedure, the light stimulus and the shock were randomly presented. One day later (vertical dotted arrow), the effects of the light stimulus on the startle response were measured. Startle probes (noise from a loudspeaker) were presented in the presence or absence of the light stimulus. Plotted are the Startle Difference Scores, i.e., the mean startle magnitude in the presence of the light minus mean startle magnitude in the absence of the light. Sample sizes are displayed below each box plot. For detailed statistical results see Supplementary Table S4. For detailed methods, see the "Materials and Methods" section in the Supplementary Presentation S1. Other details as in Figure 1.

# UNPAIRED-MEMORY IN RODENTS?

In the Section "How to Determine Baseline Odor Preferences in Larval Drosophila" we described a strategy in studying larval Drosophila that provides a baseline against which the associative effects of paired and unpaired training can be assessed - a strategy that prevents associative memories from being behaviorally expressed under baseline conditions. As mentioned, historically it was a different strategy that was applied to determine baseline behavior, namely preventing the formation of associative memories (Rescorla, 1966, 1967, 1968). In the following, we focus on fear conditioning in rodents as one study case for which that strategy has been used.

In laboratory rats or mice, non-reinforcement of a stimulus was often assumed to be mnemonically neutral and was thus used as a control in Pavlovian conditioning. For example, in differential fear conditioning a stimulus such as a tone A (serving as CS) is repeatedly paired with a punishing foot-shock reinforcement (+) (serving as US) (Ciocchi et al., 2010; Lange et al., 2014; Wigestrand et al., 2017). Intended as a control, a stimulus B, which can be a tone of another frequency, is presented in the absence of punishment. In most studies, B is presented before the beginning of the A training period (B, B, B, . . ., A+, A+, A+, etc.) (e.g., Lange et al., 2014). In fewer studies, B is presented during the A training period but unpaired from punishment (A+, B, A+, B, A+, B, etc.) (Wigestrand et al., 2017). Either way, a retention test is carried out, typically a day later. This test involves presenting A and B, in separate trials, and in a novel context. If the animals have learned the predictive relationship between A and shock, they should show freezing behavior upon the presentation of A, i.e., a species-specific defensive behavior consisting of a crouched body position and a cessation of all body movements except breathing. Provided that A and B are sufficiently distinct (Laxmi et al., 2003), freezing is observed upon presenting A but not upon presenting B, and not upon presenting a novel, not previously presented stimulus C. Does this mean that no learning about B has taken place? Not necessarily. This is because freezing is a monovalent measure, just like the PER in honeybees. In a novel context the animals hardly freeze, and thus only increases in freezing caused by negatively valenced memories can be measured. Positively valenced, unpaired memory for B, if it existed, would go unnoticed, since the animals cannot freeze less than not at all. To detect unpaired-learning, therefore, either the retardation-of-acquisition approach discussed above can be used (Pollak et al., 2010), or a bivalent measure is needed that allows positively and negatively valenced memories to be detected by modulations of the same behavioral read-out, with opposite sign. As will be discussed in the following section, up- and down-regulation of moderate levels of contextual freezing or of the startle response can provide such bivalent measures in rodents. In both cases the idea is to induce an affective state that can then either be potentiated by negatively valenced memory or attenuated by positively valenced memory.

#### Bivalent Measures of Valence in Rodents

One suggested approach to measuring positively valenced memory after unpaired training takes advantage of the contextual learning capabilities of rodents (Ostroff et al., 2010; Pollak et al., 2010; Kong et al., 2014). In these experiments, the test takes place in a context in which the animals have previously received foot-shock punishment. Within such a punishment-predicting context, the animals show the freezing behavior described above. To serve as a bivalent measure, it is important that the levels of freezing displayed by the animals should be moderate because this prevents floor or ceiling effects. If in this situation a stimulus A is presented that has itself been unpaired-trained with foot shock, the context-induced freezing is attenuated. By contrast, context-induced freezing is potentiated if A has been paired with shock. One interpretation is that through unpaired training the animals have learned that whenever A is present, punishment will not occur (a.k.a. safety learning). However, as discussed in


Section "How to Determine Baseline Odor Preferences in Larval Drosophila," a firm conclusion would require a proper baseline measure of freezing to disentangle whether the difference in freezing between paired-trained and unpaired-trained conditions results from either one of these two types of training, or from both.

A second approach takes advantage of the startle response, which can be elicited by a sudden, loud noise (a.k.a. the startle probe). This response consists of a short-latency contraction of all body muscles and can be measured by motion-sensitive devices (reviews include Koch, 1999; Fendt and Koch, 2013). To use the startle response as a read-out for a learning experiment, the animals are first trained with pairings of a stimulus A with foot-shock punishment. For the test, the startle probe is delivered either in the presence of A or in its absence. If the animals have learned the predictive relationship between A and punishment, the startle magnitude is higher in the presence than in the absence of A (a.k.a fear-potentiated startle, Davis et al., 1993; Fendt and Fanselow, 1999). This difference, i.e., startle in the presence of A minus startle in the absence of A, is quantified as the Startle Difference Score. Importantly for the present discussion, previously rewarded stimuli exert the opposite effect, i.e., startle is attenuated in their presence (Schmid et al., 1995). Thus, modulations of the startle response can be used as a bivalent measure of memory: positively valenced memories decrease startle, whereas negatively valenced memories increase it. Does this allow unpaired-memory to be revealed? Indeed, when A is presented unpaired from punishment during training, an attenuation of the startle response is observed in the test (Falls and Davis, 1994; Richardson and Fan, 2002). In this case too, one interpretation is that the animals have learned that whenever A is present, punishment will not occur (a.k.a. safety learning). However, a firm conclusion would again require a proper baseline against which to measure startle after unpaired training.

Thus, although both approaches offer bivalent measures of valence, both approaches as such also fall short of providing a proper baseline against which the effects of paired versus unpaired memory can be measured. How can such a baseline be determined?

#### Evidence for Unpaired-Memory in Rats

To determine the baseline response to a stimulus A free of associative effects of either paired or unpaired training, a procedure is needed in which no predictive relationship exists between A and punishment. To this end, Rescorla (1967) introduced the 'truly random' procedure. The idea is that A and punishment occur in a randomized temporal relationship. This means that A and punishment can also, by chance, occur together. If properly implemented, after truly random training A does not predict anything (for a more detailed discussion see Rescorla, 1972; Papini and Bitterman, 1990), whereas after unpaired training A predicts the non-occurrence of punishment (which is therefore often designated 'explicitly unpaired' training). We note that despite this critical difference in experimental outcome, the Methods sections of surprisingly many publications do not specifically state whether an unpaired

fpsyg-09-01494 August 24, 2018 Time: 12:7 # 10

For the evidence for

reinforcement-unpaired

 learning in Drosophila

 larvae mentioned

 in this study, the critical experimental

 parameters

 and site of publication

 of the original studies are presented.


TABLE 2 | Synopsis of parameters varying across experiments on innate odor preference.

For the experiments on the effects of reinforcement presence on innate odor preference in Drosophila larvae mentioned in this study, the critical experimental parameters and site of publication of the original studies are presented.

or a truly random procedure was used. This would be important, however, in order to properly interpret the results from experiments that use these procedures. In the present paper, we use 'unpaired' in the sense of explicitly unpaired throughout.

Using the truly random procedure, startle has turned out to be the same in the presence and in the absence of A, i.e., the Startle Difference Scores are zero (Davis and Astrachan, 1978; Hitchcock and Davis, 1987; Risbrough et al., 2003; Hsu et al., 2012). After unpaired training, by contrast, animals startle less in the presence of A, i.e., the Startle Difference Scores are negative (Falls and Davis, 1994; Richardson and Fan, 2002). However, neither of these studies directly compared the outcome of the two types of training. Indeed, to the best of our knowledge the first study to do so was Andreatta et al. (2012). Their data confirmed that after unpaired training startle is attenuated in the presence of A, whereas following a truly random procedure this is not the case. Critically, the Startle Difference Scores are lower after unpaired training than after the truly random procedure. As these data thus provided the first and, to our knowledge, so far the only direct evidence for unpaired-memory in rats, we here include a hitherto unpublished replication of the experiment in question, with slightly modified parameters (**Figure 5**) (see also the "Materials and Methods" section in the **Supplementary Presentation S1**). Rats were submitted to 15 presentations of a light stimulus A (5 s duration) and foot-shock punishment (0.5 s duration, 0.4 mA) with intertrial-intervals ranging between 90 and 150 s. Different groups of rats underwent one of three training conditions: (1) for one group stimulus A preceded the shock (Paired group); (2) one group received unpaired presentations of A and shock, such that the inter-stimulusinterval was never shorter than 12 s (Unpaired group); and (3) one group underwent the truly random procedure (Random group, i.e., baseline). Confirming Andreatta et al. (2012), startle was potentiated by the presence of A in the Paired group, attenuated in the Unpaired group, and unaffected in the Random group. Critically, the Startle Difference Scores were more negative in the Unpaired than in the Random group.

To summarize, from experiments using startle modulation as a bivalent behavioral read-out and the truly random procedure to determine baseline behavior, we conclude that paired and unpaired training establish oppositely valenced associative memories in rats.

## CONCLUDING DISCUSSION AND OUTLOOK

The evidence presented from larval and adult Drosophila, honeybees, and rats confirms a general principle of classical experimental psychology: that animals learn through both paired and unpaired presentations of a stimulus A with reinforcement, and that the resulting associative memories are opposite in valence. This warns against using the unpairing of A with a reward or punishment as a control for the effects of associative learning. Indeed, unpaired presentations of A and reinforcement are not a safe procedure in controlling for associative learning effects – because such a procedure can in itself establish associative memory for A as a signal that a reward or punishment will not occur.

Importantly, as we show here, this applies not only to 'absolute,' non-differential conditioning, but to differential conditioning as well: when stimulus A is presented paired with reinforcement and, in the same experimental subjects, another stimulus B is presented unpaired from reinforcement, larval Drosophila associatively learn about both stimuli – with opposite valence. Arguably, the behavior after any differential conditioning experiment might thus be a result of either or both of two types of learning process that need to be disentangled in order to fully understand the results.

Despite being established knowledge in classical psychology, the principle of opposite memories through paired and unpaired training is often neglected in neuroscience and genetics. As a consequence, the underlying mechanisms, be it on the circuit and neuronal level or the genetic and molecular level, are largely unknown. We have here presented two behavioral approaches to studying unpaired learning in two different model organisms. These approaches can now be adapted in order to unravel its underlying the mechanisms underlying unpaired learning and memory. Research in insects can play a crucial role in this endeavor. The Drosophila larva in particular has demonstrated its potential for in-depth analyses of the genetic

and neuronal mechanisms of reinforcement learning (Schroll et al., 2006; Michels et al., 2011; Rohwedder et al., 2016; Widmann et al., 2016; Eichler et al., 2017; Saumweber et al., 2018).

A full appreciation of unpaired learning would prompt a re-evaluation of the conclusions drawn from experiments comparing the effects of paired training with unpaired-control conditions, whether in differential or in non-differential 'absolute' conditioning paradigms. Although these approaches are useful to describe the outcome of associative learning in general, they cannot disentangle the effects of paired and unpaired training. If knocking-down a gene or neuronal population is found to reduce memory scores in such a task, it remains uncertain whether this gene or neuronal population is important for paired memory, or unpaired memory, or both. Likewise, if different physiological responses are elicited by a paired-trained and an unpairedtrained stimulus, it remains to be established whether effects of paired training, of unpaired training, or both are responsible for the difference. In any event, future research across species will be required to reveal whether unpaired learning as a behavioral principle is based on common mechanistic principles. If this were found to be the case, such research might help us to understand how our own behavior comes about.

#### ETHICS STATEMENT

All experiments were carried out in accordance with international guidelines for the use of animals in experiments (2010/63/EU). All experiments with rats were approved by the local ethical committee (Landesverwaltungsamt Sachsen-Anhalt, Az. 42502- 2-1309 UniMD).

#### DATA AVAILABILITY

The data for all presented behavioral experiments can be found in the **Supplementary Data Sheet S1**.

#### REFERENCES


#### AUTHOR CONTRIBUTIONS

MS, MF, and BG wrote the manuscript. SS and MS performed and analyzed the experiments displayed in **Figure 4**.

#### FUNDING

This study received institutional support from the Otto-von-Guericke-Universität Magdeburg, the Wissenschaftsgemeinschaft Gottfried Wilhelm Leibniz (WGL), the Leibniz Institute for Neurobiology (LIN), as well as grant support from the Deutsche Forschungsgemeinschaft (DFG) (GE 1091/4-1, to BG, and CRC 779 Neurobiology of motivated behavior, to BG and MF), and the European Commission grant MINIMAL (FP7 – 618045, to BG).

#### ACKNOWLEDGMENTS

Discussions with the members of our groups, as well as A. Khalili, C. König, and A. Yarali, are gratefully acknowledged, as is the continued constructive skepticism toward our conclusions by R. Menzel (Berlin). We appreciate the technical assistance of E. Kahl in performing the experiment displayed in **Figure 5**. We thank R. D. V. Glasgow (Zaragoza, Spain) for language editing.

#### SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpsyg. 2018.01494/full#supplementary-material

learning in honey bees. Front. Behav. Neurosci. 9:198. doi: 10.3389/fnbeh.2015. 00198



the application site and duration of food stimulation. Behav Neural Biol 62, 210–223. doi: 10.1016/S0163-1047(05)80019-6


Multilingual, Hands-On Manual for Odor-Taste Associative Learning in Maggots. Front. Behav. Neurosci. 11:45. doi: 10.3389/fnbeh.2017.00045


in mice. Neuropsychopharmacology 28, 654–663. doi: 10.1038/sj.npp.130 0079


memory in Drosophila larvae. PLoS Genet. 12:e1006378. doi: 10.1371/journal. pgen.1006378


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2018 Schleyer, Fendt, Schuller and Gerber. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Bimodal Patterning Discrimination in Harnessed Honey Bees

#### Breno E. Mansur, Jean R. V. Rodrigues and Theo Mota\*

Department of Physiology and Biophysics, Federal University of Minas Gerais, Belo Horizonte, Brazil

In natural environments, stimuli and events learned by animals usually occur in a combination of more than one sensory modality. An important problem in experimental psychology has been thus to understand how organisms learn about multimodal compounds and how they discriminate this compounds from their unimodal constituents. Here we tested the ability of honey bees to learn bimodal patterning discriminations in which a visual-olfactory compound (AB) should be differentiated from its visual (A) and olfactory (B) elements. We found that harnessed bees trained in classical conditioning of the proboscis extension reflex (PER) are able to solve bimodal positive and negative patterning (NP) tasks. In positive patterning (PP), bees learned to respond significantly more to a bimodal reinforced compound (AB+) than to nonreinforced presentations of single visual (A−) or olfactory (B−) elements. In NP, bees learned to suppress their responses to a non-reinforced compound (AB−) and increase their responses to reinforced presentations of visual (A+) or olfactory (B+) elements alone. We compared the effect of two different inter-trial intervals (ITI) in our conditioning approaches. Whereas an ITI of 8 min allowed solving both PP and NP, only PP could be solved with a shorter ITI of 3 min. In all successful cases of bimodal PP and NP, bees were still able to discriminate between reinforced and non-reinforced stimuli in memory tests performed one hour after conditioning. The analysis of individual performances in PP and NP revealed that different learning strategies emerged in distinct individuals. Both in PP and NP, high levels of generalization were found between elements and compound at the individual level, suggesting a similar difficulty for bees to solve these bimodal patterning tasks. We discuss our results in light of elemental and configural learning theories that may support the strategies adopted by honey bees to solve bimodal PP or NP discriminations.

Keywords: classical conditioning, bimodal learning, negative patterning, positive patterning, inter-trial interval, insect, honey bee

#### INTRODUCTION

Living in a complex world demands learning and memory of relationships between diverse stimuli in the environment. Animals can learn to associate an originally neutral stimulus (conditioned stimulus, CS) with a meaningful stimulus (unconditioned stimulus, US), an elemental association that constitutes the basis of classical conditioning (Pavlov, 1927). However, natural environments are composed by multimodal stimuli and animals usually associate these compounds with an US, rather than single unimodal elements (Lorenz, 1951). For instance, honey bees Apis mellifera use

#### Edited by:

Lars Chittka, Queen Mary University of London, United Kingdom

#### Reviewed by:

Fei Peng, Southern Medical University, China Mathieu Lihoreau, Centre National de la Recherche Scientifique (CNRS), France Nina Deisig, UMR7618 Institut d'Écologie et des Sciences de l'Environnement de Paris (IEES), France

> \*Correspondence: Theo Mota theo@icb.ufmg.br

#### Specialty section:

This article was submitted to Comparative Psychology, a section of the journal Frontiers in Psychology

Received: 31 March 2018 Accepted: 02 August 2018 Published: 24 August 2018

#### Citation:

Mansur BE, Rodrigues JRV and Mota T (2018) Bimodal Patterning Discrimination in Harnessed Honey Bees. Front. Psychol. 9:1529. doi: 10.3389/fpsyg.2018.01529

their learning capacity to exploit food sources in flowers displaying multimodal signals like colors, shapes and odors (Menzel and Mercer, 2012). Although several studies have shown that honey bees can learn to associate a single odor or color with sucrose reward (Sandoz, 2011; Menzel and Mercer, 2012), little is known about how they combine or filter relevant stimuli of distinct sensory modalities during multimodal learning tasks (Leonard and Masek, 2014).

Multimodal appetitive learning has been mostly studied using operant conditioning of free-flying bees, but results obtained so far gave rise to different conclusions. On the one hand, several studies indicated a synergistic effect between color and odor within a bimodal compound, so that combined colorodor cues led to better learning and memory compared with unimodal cues (Kunze and Gumbert, 2001; Reinhard et al., 2004, 2006; Kulahci et al., 2008). On the other hand, other studies reported inhibitory effects within a color-odor compound, so that odors tend to overshadow colors based on differences in salience (Couvillon and Bitterman, 1982, 1988, 1989; Couvillon et al., 1983; Funayama et al., 1995; Greggers and Mauelshagen, 1997). An important limitation of studies on multimodal learning in free-flying bees could be the reason of these contradictory results: differences in temporal characteristics of the two stimuli. When bees approach a color-odor cued feeder or Y-maze, color may act as a far-distance signal, and odor as a close-up signal. It is thus difficult to interpret bees' performance, given that sequential rather than simultaneous stimulus processing may occur during the approach to the target (Mota et al., 2011). These two scenarios, sequential versus simultaneous stimulus processing, may determine dramatic differences in performance, such as those supporting synergistic versus inhibitory withincompound processing.

Classical conditioning of the proboscis extension reflex (PER) in harnessed bees represents a promising alternative to study bimodal appetitive learning with a precise control of stimuli timing and duration (Gerber and Smith, 1998; Mota et al., 2011; Hussaini and Menzel, 2013). Nevertheless, such experiments are so far rare, probably because of the difficulty of training harnessed bees with visual cues (Avarguès-Weber and Mota, 2016). Whereas PER conditioning has been used for more than 50 years to study olfactory learning and memory in bees (Sandoz, 2011), only in the last two decades successful visual-PER conditioning has been achieved (Avarguès-Weber and Mota, 2016; Vieira et al., 2018). In the present work, we took advantage of this classical conditioning protocol to study bimodal patterning learning in harnessed honey bees.

Solving of patterning discriminations is considered a higherorder form of associative learning, because it involves nonlinearity and intrinsic stimulus ambiguity (Rudy and Sutherland, 1995; Giurfa, 2003). In positive patterning (PP), animals have to differentiate a reinforced compound stimulus AB+ from its nonreinforced single elements A− and B−. In negative patterning (NP), single elements A+ and B+ are reinforced whereas the compound AB− is non-reinforced (Pavlov, 1927). The nonlinearity of these patterning tasks resides in the fact that the contingency of the compound AB cannot be predicted by the simple linear summation of the contingencies of the single elements (A and B). Under these conditions, associative learning also implicates relational dependencies, as the contingencies of a given stimulus (e.g., A) vary as a function of its occurrence alone or in combination with other stimuli (stimulus ambiguity). Therefore, these patterning tasks tend to require configural learning, i.e., the ability to treat the compound stimulus as different from the simple sum of its elements (Giurfa, 2003). Whereas NP could only be solved through configural learning, PP may also be accomplished through elemental learning. According to the elemental summation principle, the associative strength of each single element in a PP task could be subthreshold for the response, but the threshold could be exceeded when both elements are combined in a compound. In NP, however, the sum of the excitatory strengths of the elements in a compound will always be higher than the strength of each single element (Deisig et al., 2001; Pearce and Bouton, 2001).

The honey bee is the only insect model that, as mammals, was shown to have the ability of solving both PP and NP tasks (Devaud et al., 2015). Studies in flies and bumble bees found that these insects can solve PP, but not NP tasks, thus suggesting their inability to accomplish configural learning problems (Young et al., 2011; Sommerlandt et al., 2014). Learning of PP and NP by honey bees has been traditionally studied using olfactory conditioning of the PER (Chandra and Smith, 1998; Deisig et al., 2001, 2002, 2007; Komischke et al., 2003; Devaud et al., 2015). The capacity to solve PP and NP was also demonstrated in free-flying honeybees trained to visual stimuli in an operant framework (Schubert et al., 2002). No study has so far analyzed in a well-controlled way the capacity of insects to solve patterning discriminations using stimuli of distinct sensory modalities, as traditionally performed in rats and rabbits (Whitlow and Wagner, 1972; Bellingham et al., 1985; Kehoe and Graham, 1988). To our knowledge, the only attempt of studying bimodal patterning learning in an insect model was made by Couvillon and Bitterman (1988). Nevertheless, these authors trained freeflying honey bees using visual and olfactory stimuli that presented distinct detection ranges and were thus perceived in a sequential way by bees (Deisig et al., 2001; Mota et al., 2011). Here, we fill this gap by training honey bees to bimodal PP and NP using equivalent duration of all stimuli, as well as simultaneous (not sequential) presentation of visual (A) and olfactory (B) elements in a compound (AB).

Previous studies showed that the temporal separation between stimuli trials (intertrial interval − ITI) clearly affects the learning performance of honey bees in olfactory patterning discrimination. Both in PP and NP tasks, increasing the ITI between conditioned trials led to better differentiation between single olfactory elements and their compound mixture (Deisig et al., 2007). So in the present study, we compare the performance of honeybees in bimodal PP and NP tasks using a shorter or a longer ITI. We also analyze memory retention to each unimodal element and the bimodal compound one hour after conditioning. Furthermore, we evaluate differences in the individual learning and memory performances of bees during these bimodal patterning tasks. Our work represents an important step to uncover the cognitive and neurobiological basis of bimodal patterning discriminations in insects.

#### MATERIALS AND METHODS

fpsyg-09-01529 August 23, 2018 Time: 9:4 # 3

#### Animals

Foragers of honeybee A. mellifera were collected from a feeder containing 30% (v/v) sugar solution 50 meters from six outdoor hives kept in the Ecological Station of the Federal University of Minas Gerais (UFMG, Brazil). All experiments were conducted in the Brazilian spring/summer season (from September to March). Bees were placed in small glass vials, cooled on ice until they ceased their movements and then harnessed in plastic tubes using thin pieces of soft masking tape. The wings were protected by a piece of filter paper. Each bee was fed 1 µl of 30% (v/v) sugar solution after fixation and then kept for one hour in a dark chamber with high humidity.

#### Conditioned and Unconditioned Stimuli

Visual CS (A) consisted of an illuminated 20 × 20 cm screen covered with a chromatic transmission filter (LF124S Dark Green: peak at 535 nm or LF119S Dark Blue: peak at 455 nm; LEE Filters) and tracing paper for light dispersion. A white-LED light source (E27-5W Cool White; Epistar) connected to a linear potentiometer provided illumination with controlled intensity behind the colored screen. Taking into account the spectral sensitivities of the honeybee photoreceptors (Peitsch et al., 1992), the green stimulus excited 0, 15, and 85% of the short- (S), medium- (M), and large-range (L) wavelength photoreceptors, respectively. For the blue stimulus, these values were 2, 68, and 30%, respectively. The large-field colored screen was placed at a distance of 10 cm from the bee right eye, so that it subtended a visual angle of 90◦ . The irradiance of blue or green stimulus was adjusted to 0,4 µW cm−<sup>2</sup> at the level of the bee eye by using a spectrophotometer (USB2000 + UV-VIS-ES, Ocean Optics) radiometrically calibrated using a deuterium/tungsten light source (DH-2000-BAL, 220−1050 nm, Ocean Optics). Absolute irradiance was measured using an optical fiber (QP600-2-UV-VIS, Ocean Optics) coupled to a cosine corrector with Spectralon diffusing material (CC-3-UV-S, Ocean Optics).

Olfactory CS (B) was 2-hexanol or 1-nonanol (Sigma-Aldrich, Brazil). Five microliters of pure odorant were applied onto a 1 cm<sup>2</sup> stripe of filter paper placed into a 30 mL syringe, which allowed frontal odorant delivery to the antennae. An air extractor placed behind the bee prevented odorant accumulation.

The US was 1 µL of 30% (w/w) sugar solution delivered to the bee by means of a micropipette.

The reason we presented a lateral screen stimulating a single eye instead of a frontal one stimulating both eyes was the fact that both the syringe used to deliver the odor and the micropipette used to deliver sugar solution were already presented in a frontal position. When odor, color and reward overlapped during patterning conditioning, the syringe and the micropipette produced large shades on the visual screen. These shades may be used by the bees as conditioned or secondary stimulus. We thus decided to present visual stimulation only to the right eye, because a previous study on visual conditioning of the PER indicates that honey bees learn better in this framework using the right than the left eye (Letzkus et al., 2008). After this work, other authors confirmed that lateral stimulation of the right eye is an efficient method for training harnessed bees to visual stimuli (e.g., Niggebrügge et al., 2009; Vieira et al., 2018).

# Experimental Setup and Conditioning Procedure

All experiments were performed in a dark room illuminated by a low intensity red-light source (peak at 660 nm). During conditioning, the plastic tube holding the bee was tilted to 45◦ and fixed in a platform of 9 cm high (Vieira et al., 2018). In this position, the right eye of the bee was at a distance of 10 cm from the center of the visual stimulation screen. In PP experiments, presentation of visual or olfactory stimulus alone was not rewarded whereas their simultaneous presentation (compound stimulus) was rewarded (A−, B−, and AB+). In NP experiments, individual presentations of the visual or olfactory stimulus were rewarded whereas the compound bimodal stimulus was not (A+, B+, and AB−). Both in PP and NP, training consisted of 10 trials of each stimulus (A, B, and AB), thus totalizing 30 trials presented in a pseudorandom sequence starting with A, B or AB in a balanced way. At most, two trials of a same stimulus followed each other during conditioning.

At the beginning of each rewarded trial the bee was placed in the conditioning setup for 30 s to allow familiarization with the experimental context. Thereafter, CS+ (A, B or AB) was presented for 7 s. Four seconds after the onset of the CS+, the US was delivered to the bee for 3 s. Therefore, the interstimulus interval (ISI) was 4 s and the overlap between CS and US was 3 s. The bee was removed from the setup 23 s after reward, thus completing a total of 60 s per trial. Unrewarded trials followed the same time sequence, but stimulation was not paired with reward. To analyze the effect of inter-trial interval (ITI) on bimodal PP and NP, we trained two independent groups in each of these paradigms with an ITI of 3 and 8 min, respectively. Training with 3 min ITI was performed using green 535 nm as visual stimulus and 2-hexanol as olfactory stimulus (N = 45 bees, both for PP and NP). Training with 8 min ITI was performed using this same pair of stimuli (green 535 nm and 2-hexanol) or an alternative pair consisted of blue 455 nm and 1-nonanol (N = 45 bees/pair of stimuli, both for PP, and NP). One hour after the end of conditioning, all experimental groups were submitted to retention tests consisted of an unrewarded presentation of each stimulus (A, B, or AB) with an ITI equivalent to that used during training (3 min or 8 min ITI). The order of presentation of the three stimuli during retention tests was randomized between subjects in all experimental groups.

The beginning and the end of each trial, as well as the onset and offset of CS and US were signaled by a computer programmed to emit tones of different frequencies for each event. The occurrence of proboscis extension was recorded within the first 4 s of CS presentation (conditioned response), as well as during the US presentation. Animals that did not show PER for more than 3 times during the US presentation (<5%) were excluded from our analysis, as they may present impairment of muscular reflex and/or sucrose responsiveness.

#### Statistical Analysis

fpsyg-09-01529 August 23, 2018 Time: 9:4 # 4

Two-way analysis of variance in generalized linear model (GLM) for repeated measures was used to analyze within (stimulus × trial effect) and between group (group × stimulus × trial effect) performances in PP and NP conditioning. Further Tukey's multiple comparisons were used to analyze differences between: (i) responses to each stimulus; (ii) performances in different trials of conditioning. One-way GLM for repeated measures followed by Tukey's multiple comparisons was used to compare responses to each stimulus in retention tests. The alpha level was set to 0.05 (two tailed) for all analyses. All statistical analyses were conducted using the software IBM SPSS Statistics 21.0.

#### RESULTS

## Bimodal Patterning Discrimination With 3 min ITI

Honeybees trained to discriminate a visual element (A−) and an olfactory element (B−) from its bimodal compound (AB+) in a PP protocol with an ITI of 3 min were successful in learning the task. **Figure 1A** shows the percentage of PER along 10 trials of each stimulus and reveals significant differences between the learning curves of A−, B−, and AB+ (stimulus × trial GLM for repeated measures; stimulus effect: F2,<sup>88</sup> = 19.6, p < 0.001; interaction: F18,<sup>792</sup> = 5.4, p < 0.001). During the first five trials, bees showed equivalent levels of increasing response to the olfactory element (B−) and the compound (AB+) whereas responses to the visual element (A−) remained much lower. After the fifth trial, however, responses to B− started to decrease whereas responses to AB+ kept increasing until the end of conditioning (**Figure 1A**). Although global performances significantly differed between all stimuli (Tukey test; stimulus effect; A− vs. B− and A− vs. AB+: p < 0.001; B− vs. AB+: p < 0.05), no differences were found in responses to A− and B− at the last trial of conditioning (Tukey test; stimulus × trial 10 effect; A− vs. B−: NS). Moreover, bees responded significantly more in the last trial to the compound AB+ than to the elements A− and B− (Tukey test; stimulus × trial 10 effect; A− vs. AB+ and B− vs. AB+: p < 0.001), thus confirming successful PP solving. In retention tests performed one hour after conditioning (**Figure 1A**), bees were again able to discriminate each unimodal element from the bimodal compound (GLM for repeated measures; F2,<sup>88</sup> = 20.2, p < 0.001; Tukey test; A− vs. B−: NS; A− vs. AB+ and B− vs. AB+: p < 0.001).

While honeybees were successful in learning a bimodal PP task with an ITI of 3 min (**Figure 1A**), this was not the case for a NP task with the same ITI (**Figure 1B**). In this patterning approach, bees showed increasing PER to all three stimuli along trials, with equivalent levels of response to the olfactory element B+ and the bimodal compound AB− (stimulus × trial GLM for repeated measures; stimulus effect: F2,<sup>88</sup> = 25.9, p < 0.001; interaction: F18,<sup>792</sup> = 1.8, p < 0.05; Tukey test; stimulus effect; A+ vs. B+ and A+ vs. AB−: p < 0.01; B+ vs. AB−: NS). The comparisons between the responses to each stimulus at the last conditioning trial also show that bees were unable to differentiate B+ from AB− in this NP task (Tukey test; stimulus × trial 10 effect; A+ vs. AB+ and A+ vs. AB− : p < 0.05; B+ vs. AB−: NS). Retention tests performed one hour after conditioning (**Figure 1B**) also confirm an absence of discrimination between the olfactory element and the bimodal compound (GLM for repeated measures; F2,<sup>88</sup> = 11.0, p < 0.001; Tukey test; A+ vs. B+ and A+ vs. AB−: p < 0.05; B+ vs. AB−: NS).

#### Bimodal Patterning Discrimination With 8 min ITI

**Figure 2A** shows the performance of bees trained to a PP task using the same visual (green 535 nm) and olfactory stimuli (2-hexanol) as in **Figure 1A**, but with a longer ITI of 8 min. As previously observed (**Figure 1A**), bees started the task with similar increasing response levels to B− and AB+, but after the fifth trial they begun to discriminate these stimuli (stimulus × trial GLM for repeated measures; stimulus effect: F2,<sup>88</sup> = 10.8, p < 0.001; interaction: F18,<sup>792</sup> = 3.6, p < 0.001; Tukey test; stimulus effect: A− vs. B− and A− vs. AB+: p < 0.001; B− vs. AB+: p < 0.01). Responses at the last conditioning trial significantly differed between each unimodal element and the bimodal compound, but not between the visual and olfactory elements (Tukey test; stimulus × trial 10 effect; A− vs. B−: NS; A− vs. AB+ and B− vs. AB+: p < 0.001). Together with these results, responses of bees during retention tests (**Figure 2A**) confirmed successful bimodal PP solving (GLM for repeated measures; F2,<sup>88</sup> = 20.6, p < 0.001; Tukey test; A− vs. B−: NS; A− vs. AB+ and B− vs. AB+: p < 0.001).

We simultaneously trained another group of bees to the same PP task with 8 min ITI, but using alternative visual (blue 455 nm) and olfactory stimuli (1-nonanol). The performance of bees in this experimental group (**Figure 2B**) was very similar to the one of bees trained using green 535 nm and 2-hexanol as conditioned stimuli (**Figure 2A**). They were able to discriminate each unrewarded unimodal element from the rewarded bimodal compound both during conditioning (stimulus × trial GLM for repeated measures; stimulus effect: F2,<sup>88</sup> = 17.5, p < 0.001; interaction: F18,<sup>792</sup> = 5.5, p < 0.001; Tukey test; stimulus × trial 10 effect; A− vs. B−: NS; A− vs. AB+ and B− vs. AB+: p < 0.001) and retention tests (GLM for repeated measures; F2,<sup>88</sup> = 19.4, p < 0.001; Tukey test; A− vs. B−: NS; A− vs. AB+ and B− vs. AB+: p < 0.001). Since we found no statistical differences between these two experimental groups (**Figures 2A,B**) trained using distinct pairs of stimuli (group × stimulus × trial GLM for repeated measures; group effect: F1,<sup>88</sup> = 1.3, NS), we pooled results from each of them in a single graphic (**Figure 2C**). As expected, all statistical effects in this pooled group (**Figure 2C**) were equivalent to those described for experimental groups presented in **Figures 1A,B** both during conditioning (stimulus x trial interaction: F18,<sup>1584</sup> = 6.7, p < 0.001; Tukey test; stimulus × trial 10 effect; A− vs. B−: NS; A− vs. AB+. and B− vs. AB+: p < 0.001) and retention tests (GLM for repeated measures;

discrimination tasks with an ITI of 3 min. Conditioning consisted of 10 trials of each stimulus presented in a pseudorandom sequence (left). One hour after conditioning, unrewarded retention tests were performed for each stimulus (right). (A) Positive patterning (PP; N = 45). (B) Negative patterning (NP; N = 45). Asterisks indicate significant differences in GLM followed by Tukey test comparing responses to each stimulus in the last conditioning trial. Different lowercase letters (x,y) indicate significant differences in GLM followed by Tukey test comparing responses to each stimulus during retention tests.

F2,<sup>178</sup> = 36.3, p < 0.001; Tukey test; A− vs. B−: NS; A− vs. AB+ and B− vs. AB+: p < 0.001).

**Figure 3** shows the performances of honeybees trained in a bimodal NP task with 8 min ITI using green 535 nm and 2 hexanol (group A; **Figure 3A**) or blue and 1-nonanol (group B; **Figure 3B**) as pair of stimuli. Different from bees trained in a NP task with 3 min ITI (**Figure 1B**), we found that the larger ITI of 8 min allowed honeybees to solve a bimodal NP task. In both experimental groups (**Figures 3A,B**), bees started the task by increasing their responses to all three stimuli, but begun to decrease their levels of response to the unrewarded bimodal compound after the fourth or fifth trial. At the end of conditioning, both groups were able to discriminate each unimodal element from its bimodal compound (stimulus × trial GLM for repeated measures; stimulus effect; group A: F2,<sup>88</sup> = 11.8, p < 0.001; group B: F2,<sup>88</sup> = 16.7, p < 0.001; interaction; group A: F18,<sup>792</sup> = 5.9, p < 0.001; group B: F18,<sup>792</sup> = 6.7, p < 0.001; Tukey test; stimulus × trial 10 effect; both groups: A+ vs. AB−: p < 0.05; B+ vs. AB−: p < 0.001). Different from bees trained to bimodal PP tasks (**Figures 1A**, **2**), levels of response significantly differed at the end of conditioning between the visual and the olfactory elements (**Figures 3A,B**; Tukey test; stimulus × trial 10 effect; both groups: A+ vs. B+: p < 0.05). In retention tests performed one hour after conditioning (**Figures 3A,B**), levels of response were significantly different between all three stimuli (GLM for repeated measures; group A: F2,<sup>88</sup> = 17.1, p < 0.001; group B: F2,<sup>88</sup> = 23.2, p < 0.001; Tukey test; both groups: A+ vs. B+: p < 0.05; A+ vs. AB−: p < 0.05; B+ vs. AB−: p < 0.001).

We found no statistical differences between performances of these two experimental groups shown in **Figures 3A,B**

(group × stimulus × trial GLM for repeated measures; group effect: F1,<sup>88</sup> = 1.1, NS), thus we pooled their results in a single graphic (**Figure 3C**). Statistical effects in this pooled group (**Figure 3C**) were equivalent to those described for experimental groups presented in **Figures 3A,B** both during acquisition (stimulus × trial interaction: F18,<sup>1584</sup> = 11.2, p < 0.001; Tukey test; stimulus × trial 10 effect; A+ vs. B+ and A+ vs. AB−: p < 0.01; B+ vs. AB−: p < 0.001) and retention tests (GLM for repeated measures; F2,<sup>178</sup> = 37.0, p < 0.001; Tukey test; A+ vs. B+ and A+ vs. AB−: p < 0.01; B+ vs. AB−: p < 0.001). In conclusion, while an ITI of 8 min allowed solving both PP (**Figure 2**) and NP (**Figure 3**), only PP could be solved with a shorter ITI of 3 min (**Figure 1**).

# Distinct Learning Categories in Bimodal Patterning Solving

Considering that responses to element and compound stimuli during retention tests reflected very well the level of discrimination reached in bimodal PP (**Figure 2**) or NP (**Figure 3**), we analyzed these responses at the individual level to classify bees into distinct learning categories. In retention tests, bees could respond or not only once to each of the three stimuli (A, B, and AB), thus eight different combinations of response may emerge (000, 111, 100, 010, 001, 011, 110, 101; to A, B, and AB, respectively). Successful learners of a PP task should not respond to the unrewarded A and B elements, and respond to the rewarded compound AB (001). Successful learners of NP should respond to the rewarded elements A and B, and not respond to the unrewarded compound AB (110). We thus asked: what is the proportion of bees presenting successful performances in retention tests after bimodal PP and NP? Which are the other categories of response emerging during these tasks? How do the learning curves of bees in these different categories look like? To answer these questions we analyzed the individual response of 90 bees trained to bimodal PP (**Figure 2C**) or NP (**Figure 3C**) with an ITI of 8 min.

**Figure 4** shows the three major learning categories emerging in bees trained to bimodal PP, classified according to their responses in retention tests. From 90 bees (**Figure 2C**), only 24 (27%) presented exactly correct responses (001) in retention tests (**Figure 4A**). Surprisingly, almost half of the bees (n = 42; 47%) responded equally to all three stimuli (000 or 111), thus presenting a generalist strategy toward unimodal elements and compound stimuli (**Figure 4B**). The third major category of response (n = 21; 23%) emerging during bimodal PP solving consisted of individuals not responding to the unrewarded visual element A, but responding to the unrewarded olfactory element B and the compound stimulus AB (011; **Figure 4C**). Only three bees (3%) could not be classified in one of these three learning categories (**Figure 4D**). Two of them responded only to the olfactory element B (010), whereas one bee responded to A and AB, but not to B (101).

Bees classified as good PP learners (**Figure 4A**) clearly solved the task, but as observed in the overall performance of bees trained to PP (**Figure 2C**), they presented increasing responses to the unrewarded odor (B) at the beginning and around the fifth trial started to suppress these responses. At the end of conditioning, these bees noticeably discriminate between each unrewarded unimodal element and the rewarded bimodal compound (stimulus × trial GLM for repeated measures; interaction: F18,<sup>414</sup> = 9.3, p < 0.001; Tukey test; stimulus × trial 10 effect; A− vs. B−: NS; A− vs. AB+, and B− vs. AB+: p < 0.001). On the other hand, bees presenting generalist responses (**Figure 4B**) were completely unable to discriminate between any of the stimuli during bimodal PP (stimulus × trial GLM for repeated measures; interaction: F18,<sup>738</sup> = 1.4, NS). The last learning category observed in bimodal PP (**Figure 4C**) was composed by bees that discriminate between the visual element A and the other stimuli, but could not differentiate the olfactory element B from the bimodal compound AB (stimulus × trial GLM for repeated measures; interaction: F18,<sup>342</sup> = 2.6, p < 0.001; Tukey test; stimulus × trial 10 effect; A− vs. B− and A− vs. AB+: NS; B− vs. AB+: NS).

All 90 bees trained to bimodal NP (**Figure 3C**) could be classified into one of the following four learning categories according to their responses in retention tests: good NP learners (110; **Figure 5A**); generalists (000 or 111; **Figure 5B**); responding only to B (010; **Figure 5C**); responding to B and AB (011, **Figure 5D**). Bees classified as good NP learners (**Figure 5E**; N = 22; 25%) were clearly able to discriminate between each rewarded unimodal element and the unrewarded bimodal compound (**Figure 5A**; stimulus × trial GLM for repeated measures; interaction: F18,<sup>378</sup> = 7.0, p < 0.001; Tukey test; stimulus × trial 10 effect; A+ vs. B+: NS; A+ vs. AB−, and B+ vs. AB−: p < 0.001). As well as in bimodal PP (**Figure 4B**), a large amount of bees confronted to a bimodal NP task (**Figure 5E**; N = 40, 44%) developed a generalist strategy and were totally unable to discriminate between the three stimuli (**Figure 5B**; stimulus × trial GLM for repeated measures; interaction: F18,<sup>702</sup> = 1.5, NS).

The third most representative learning category observed in bimodal NP (**Figure 5E**; N = 18; 20%) was composed by bees that discriminate between the olfactory element B and the other stimuli, but could not differentiate the visual element A from the bimodal compound AB (**Figure 5C**; stimulus × trial GLM for repeated measures; interaction: F18,<sup>306</sup> = 6.7, p < 0.001; Tukey test; stimulus × trial 10 effect; A+ vs. B+ and B+ vs. AB−: p < 0.001; A+ vs. AB−: NS). Finally, the fourth learning category emerging in bimodal NP (**Figure 5E**; N = 10; 11%) was composed by bees that presented low response levels to the rewarded visual element A, but developed increasing responses to the rewarded element B and the unrewarded compound AB (**Figure 5D**; stimulus × trial GLM for repeated measures; interaction: F18,<sup>162</sup> = 1.9, p < 0.05; Tukey test; stimulus × trial 10 effect; A+ vs. B+ and A+ vs. AB−: p < 0.001; B+ vs. AB−: NS).

# DISCUSSION

Our study shows that honey bees are able to solve bimodal PP and NP in a classical PER conditioning protocol using a colored-light screen as visual stimulus and a pure synthetic odor as olfactory stimulus. The two pairs of visual-olfactory

stimuli used in our experiments (green 535 nm/2-hexanol or blue 455 nm/1-nonanol) induced equivalent levels of bimodal patterning discrimination, both in PP and NP. While an ITI of 8 min allowed solving both PP and NP tasks, only PP could be solved with a shorter ITI of 3 min. This result agrees with previous experiments on olfactory patterning discrimination that found better performances using longer trial-spacing (Deisig et al., 2007). More precisely, these authors found that honeybees trained in olfactory PER conditioning were unable to solve PP or NP with ITIs of 1 min or 3 min, whilst an ITI of 5 min allowed solving only PP. As well as in our bimodal patterning approach, an ITI of 8 min favored solving of both olfactory PP and NP (Deisig et al., 2007). Many reasons can account for the fact that an ITI of 3 min allowed bimodal PP solving in our work, but not olfactory PP solving in that previous study: nature of the stimuli (visual-olfactory vs. only olfactory); number of trials (10 per stimulus vs. 4 per element and 8 for the compound); duration of CS presentation (7 s vs. 6 s); experimental context (dark room and 45% body inclination vs. illuminated room and vertical body position), among others. Altogether, results on trial-spacing effect in patterning solving by bees are in line with an extensive literature showing that animals often present better learning when CS trials are temporally more spaced (Gibbon et al., 1977; Barela, 1999; Sunsay et al., 2004).

Previous studies in olfactory patterning discrimination by bees suggest that a balanced proportion of reinforced and nonreinforced trials (1:1 CS+ /CS− rate) favors discrimination in those tasks (Deisig et al., 2001, 2007). In the new bimodal conditioning approach here developed, we had an important limitation to develop PP and NP with such a contingency balance: reasonable levels of visual learning by harnessed bees are only reached with a large amount of trials (Avarguès-Weber and Mota, 2016). If we performed 10 trials for each element and 20 trials for the bimodal compound (1:1 contingency rate) with an ITI of 8 min, our conditioning protocol together with the memory test would least more than six hours. Considering also the time for capturing and harnessing the bees, as well as the one hour resting period prior to conditioning, it was simply impossible for us to perform such an experiment. We actually tried to perform a protocol using five trials of each element and 10 trials of

the compound with an ITI of 8 min, but the levels of learning obtained for the visual element were very poor and there was no successful discrimination in PP or NP (data not shown).

The fact that only a bimodal PP task could be solved with a shorter ITI of 3 min, but not a NP task is in agreement with several studies observing that PP may be learned using a different strategy than NP (e.g. Rescorla, 1972; Bellingham et al., 1985; Kehoe, 1988; Deisig et al., 2007). Solving of NP tasks in rabbits requires longer CS and ITI duration than PP tasks (Kehoe and Graham, 1988; Kinder and Lachnit, 2002). In humans, longer processing time was found in response to the compound stimulus during NP when compared to PP (Lachnit et al., 2002).

Interestingly, olfactory NP solving in bees requires olfactory input from both the antennae, whereas PP can be solved with unilateral olfactory stimulation of a single antennae (Komischke et al., 2003). Together with our data, these results in different models support the assumption that NP is solved using a learning strategy that requires different resources than the ones employed to solve PP.

It has been suggested that PP solving admits an elemental learning strategy, whereas NP solving can exclusively rely in a configural learning (non-elemental) strategy (Pearce, 1994; Deisig et al., 2001; Giurfa, 2003; Devaud et al., 2015). In the present work, we analyzed the individual performances of 90 honey bees in bimodal PP or NP, and we found that different learning strategies emerged in both these paradigms (see section "Distinct learning categories in bimodal patterning solving"). The first surprising observation from this analysis was the high percentage of bees that presented strong generalization between all stimuli and were thus unable to solve the discrimination both in the PP and the NP approach (47 and 44% of bees, respectively). These results highlight the level of difficulty of these tasks and rule out the possibility that bees solve them using an extreme configural learning strategy (Williams and Braker, 1999). These extreme configural theories are different from Pearce's configural theory (Pearce, 1994), because they predict no generalization between a compound and its elements, since the compound would be treated as a totally new stimulus completely unrelated to its elements (Williams and Braker, 1999; Deisig et al., 2003).

Apart from bees that completely generalized between stimuli in PP or NP, we also found intermediate learning categories that presented generalization between one of the elements and the compound. In the case of PP, generalization between the olfactory element and the compound occurred in 23% of bees. Bees in this learning category seemed to reduce the complexity of the problem by treating it as an elemental differential conditioning task (A− vs. AB/B+). The olfactory element B and the compound AB appeared to be both treated as one rewarded odor. In the case of NP, generalization between one of the elements and the bimodal compound was found not only for the olfactory element B as in the bimodal PP task, but also for the visual element A. Curiously, a representative amount of individuals trained to bimodal NP (20%) generalized between the visual element A and the compound AB, as if they solved the task following a differential conditioning schedule (A/AB− vs. B+). Furthermore, a smaller amount of bees (11%) trained to NP, developed generalized responses between the olfactory stimulus B and the compound AB that were similar to the ones observed in some bees trained to PP (A− vs. AB/B+). In this case, however, bees were twice wrong in their responses, because they were supposed to respond to A+ and not respond to AB− in the NP task. Although different categories of generalization emerged in bimodal PP (to all stimuli; between odor and compound) and NP (to all stimuli; between odor and compound; between color and compound), the equivalent total amount of unsuccessful bees in these tasks (63 and 65%, respectively) suggests a similar difficulty for bees to solve them.

Although most of the bees trained to bimodal PP or NP task presented generalization and were unsuccessful in solving the task, we also identified a category of very efficient learners in both these approaches (27 and 25% of bees, respectively). Our results, therefore, reinforce the notion that averaged learning curves and memory retention scores obtained from a group of animals often hide a more elaborate range of learning dynamics that can only be observed at the individual level (Gallistel et al., 2004). Accordingly, several recent studies on learning and memory by bees emphasize the importance of better analyzing the dynamics of individual performances (e.g., Mota and Giurfa, 2010; Dobrin and Fahrbach, 2012; Pamir et al., 2014; Evans et al., 2017; Vieira et al., 2018). Similar to our results on bimodal patterning, only 30% of 111 bees trained to a multiple olfactory reversal task were able to accurately solve this non-elemental problem (Mota and Giurfa, 2010). An extensive evaluation of individual performances of 3298 bees in different elemental olfactory tasks also revealed that group-averaged learning analysis hid drastic inter-individual differences (Pamir et al., 2014). Altogether, these studies indicate that finding a single associative learning theory to explain the averaged performance of a population is tricky, and may not reflect the real complexity of CS and US representations at the individual level. Effective learners of bimodal PP may use elemental or configural strategies to solve this task, whereas good learners of bimodal NP could only use configural strategies. Finding which of those alternative accounts is actually used by bees remains so far a challenge, even for the more studied unimodal olfactory PP and NP in honey bees (Deisig et al., 2003, 2007; Devaud et al., 2015).

Although individual bees trained to elemental olfactory tasks differ in terms of the number of trials required to develop the first conditioned response (CR), as well as the stability of the CR along trials, high levels of success are usually reached at the end of conditioning (Pamir et al., 2014). Pamir and collaborators found that 54% of the responsive animals trained to elemental olfactory tasks already developed conditioned responses (CR) to the CS+ after a single trial of conditioning. By the third trial about 80% of those animals presented correct responses to the CS+. They also show a high average level of CR stability once the animal starts to respond. The average percentage of nonresponding animals in elemental olfactory tasks was only ∼20%. All in all, this analysis reveals high levels of individual success (∼80%) on the solving of elemental forms of associative learning by harnessed bees (Pamir et al., 2014). On the other hand, only 25 to 30% of harnessed bees were able to solve a non-elemental olfactory task (Mota and Giurfa, 2010) or a bimodal patterning discrimination (present study). These studies indicate that honey bees present much higher rates of success in solving elemental than non-elemental tasks in a classical conditioning framework. Moreover, the equivalent low levels of success obtained in bimodal PP and NP tasks in our study suggests that PP is probably solved by bees using an non-elemental (configural) learning strategy.

Previous studies in non-human mammals suggested that the use of different sensory modalities in patterning tasks may favor the emergence of elemental rather than configural strategies (e.g., Redhead and Pearce, 1995; Brandon et al., 2000; Myers et al., 2001). The relative salience of the elements is also an important feature that can influence the associations acquired in

patterning discriminations, particularly in NP tasks (Delamater, 2012). For instance, when stimuli with different saliences were used in NP tasks, discrimination was first learned between the unrewarded compound and the less salient element, as compared with the more salient element (Redhead and Pearce, 1995; Delamater et al., 1999). Studies on bimodal learning by honey bees have often suggested that odors are more salient cues than colors (e.g. Couvillon and Bitterman, 1989; Funayama et al., 1995; Greggers and Mauelshagen, 1997), especially in harnessed individuals (Gerber and Smith, 1998; Mota et al., 2011). Probably for that reason, learning levels acquired in visual PER conditioning are typically lower than those reported for olfactory PER conditioning (Avarguès-Weber and Mota, 2016), as also observed in the present study. The higher salience of odors over colors in our bimodal patterning approach may be responsible for the strong generalization observed between the olfactory element B and the compound AB, as well as the better levels of discrimination between the visual element A and the compound AB, in certain cases.

A recent study showed that the mushroom bodies (MBs) of the honey bee brain are necessary for solving both olfactory PP and NP (Devaud et al., 2015). Pharmacological inhibition of the MBs disrupted the capacity of bees to solve PP and NP, but not their ability to learn elemental olfactory discriminations. Therefore, apart from the well-known role of the MBs in memory storage and retrieval, theses insect brain structures seem to be implicated in the acquisition of ambiguous olfactory discrimination problems (Devaud et al., 2015). The necessity of the MBs for solving olfactory PP and NP, but not elemental olfactory discriminations, strongly suggests that olfactory PP is solved by bees using a non-elemental rather than an elemental summation strategy (Devaud et al., 2015). Little is known, however, about the role of the MBs on elemental and nonelemental visual or bimodal learning in bees. It might be that the simple necessity of integrating visual and olfactory information for bimodal patterning learning would require the integrative role of the MBs.

The MBs are indeed the main region of the honey bee brain where a convergence between visual and olfactory neural circuits was clearly identified (Mobbs, 1982; Ehmer and Gronenberg, 2002). Considering that a cross-modal interaction between

#### REFERENCES


olfactory and visual cues is necessary to solve bimodal PP and NP, the MBs appear as the most probable structures mediating these discriminations. A recent study in Drosophila found that visual and olfactory associative learning share dopaminergic neural circuits in the MBs, confirming that distinct sensory memories are processed in this common brain center (Vogt et al., 2014). Alternative regions for cross-talk between visual and olfactory circuits in the bee brain have also been suggested in the median, lateral, and posterior protocerebrum (Erber and Menzel, 1977; Maronde, 1991), but the role of these structures in learning and memory remains poorly understood. Future studies should combine the new bimodal PP and NP protocols here presented to pharmacological or neurophysiological techniques, in order to uncover the neural mechanisms underlying these cognitive phenomena.

## AUTHOR CONTRIBUTIONS

TM conceived the study and designed the methodology. BM, JR, and TM performed the experiments and analyzed the data. BM and TM wrote the first draft of the manuscript. All authors contributed to the final version of the manuscript.

# FUNDING

We thank the support of Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq: 457718/2014-5 to TM) and Fundação de Amparo à Pesquisa do Estado de Minas Gerais (FAPEMIG: APQ-02013-13 to TM). JR received a scholarship from the Program Young Talent of Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES/Science without Borders – Brazilian Government).

# ACKNOWLEDGMENTS

We are thankful to Ivan Gastelois and the Ecological Station of the Federal University of Minas Gerais for valuable assistance in beekeeping.




Young, J. M., Wessnitzer, J., Armstrong, J. D., and Webb, B. (2011). Elemental and non-elemental olfactory learning in Drosophila. Neurobiol. Learn. Mem. 96, 339–352. doi: 10.1016/j.nlm.2011.06.009

**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2018 Mansur, Rodrigues and Mota. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Does Cognition Have a Role in Plasticity of "Innate Behavior"? A Perspective From Drosophila

#### E. Axel Gorostiza\*

Departamento de Farmacología, Facultad de Ciencias Químicas, Instituto de Farmacología Experimental de Córdoba-CONICET, Universidad Nacional de Córdoba, Córdoba, Argentina

Keywords: innate behavior, insects, behavioral flexibility, Drosophila, stimulus-response, behavioral plasticity

# BEHAVIORAL PLASTICITY AND THE STIMULUS–RESPONSE MODEL

The term innate is commonly used to refer to behaviors inherited and not learned or derived from experience. This definition denies or ignores the inborn components of learning. An animal can only learn if it already has the components required for learning, e.g., the molecular and neuronal substrates. Moreover, all behaviors are, to some extent, susceptible to modification by experience. Hence, no behavior can be strictly learned or innate (Shettleworth, 2010), making this distinction and the terms scientifically inappropriate to some extent (Mameli and Bateson, 2006; Bateson and Mameli, 2007). However, given the absence of a better term, it is still possible to find some behaviors classified as innate behaviors in literature, and defined as "stereotypic patterns of movement inherited from birth that require no prior experience for proper execution" (Kim et al., 2015). Noticeably, the concept of stereotypy was arbitrarily included into the definition of innate behavior. This group includes behaviors as different as escape responses (movements performed by an animal to avoid a possible predator; Card, 2012), taxes (orienting movement of an organism directed in relation to a stimulus; Zupanc, 2010), and courtship. What these behaviors have in common is that they are dominated by innate components and preferences and seem to be stereotypic and automatic responses elicited by a defined stimulus (i.e., reflexes, senso Purves et al., 2004). They are considered sensory-motor routines driven by inborn responses to biologically relevant sensory cues. This is the base of the sensory-response model, wherein the brain only reacts to external stimuli and the behaviors are the responses (Dickinson, 1985). Although there is increasing evidence of an active role of the brain with the external stimuli exerting only a modulatory effect in humans (Raichle, 2010) and invertebrates (Gaudry and Kristan, 2009; Gordus et al., 2015), many innate behaviors in insects are still described using the sensory-response model. This interpretation led to some important aspects of innate behaviors being neglected or misinterpreted. Any behavioral researcher has experienced that these behavioral responses are far from constant among groups or single individuals from the same species, or even the same retested individual (Kain et al., 2012; Buchanan et al., 2015). Researchers work hard to control this behavioral variability by modifying their experiments. Some of these manipulations include only using animals in a certain internal or motivational state. For example, in olfactory appetitive learning in Drosophila, only starved animals are used and the length of the starvation period influences the results (Colomb et al., 2009). Similarly, when using the proboscis extension reflex assay, animals that did not respond to sucrose in a pretraining session or naïve animals that displayed spontaneous proboscis extension to water are discarded (Shiraiwa and Carlson, 2007). However, in our efforts to control the response of animals, we are probably curtailing the repertoire of actions we can observe, thus imposing the response we want to study onto our results. This increases the probability of that specific actions occurring and, importantly, lead us to forget the importance of variability for survival. An

#### Edited by:

Lars Chittka, Queen Mary University of London, United Kingdom

#### Reviewed by:

Klaus Lunau, Heinrich Heine Universität Düsseldorf, Germany Benjamin L. De Bivort, Harvard University, United States

> \*Correspondence: E. Axel Gorostiza eagorostiza@unc.edu.ar orcid.org/0000-0002-0185-971X

#### Specialty section:

This article was submitted to Comparative Psychology, a section of the journal Frontiers in Psychology

Received: 01 March 2018 Accepted: 30 July 2018 Published: 31 August 2018

#### Citation:

Gorostiza EA (2018) Does Cognition Have a Role in Plasticity of "Innate Behavior"? A Perspective From Drosophila. Front. Psychol. 9:1502. doi: 10.3389/fpsyg.2018.01502 automatic and rigid response could soon be disadvantageous. What is an appropriate response to a given stimulus when the animal is hungry may be maladaptive when the animal is seeking a mating partner or escaping from a predator, and vice versa. The animal must evaluate its internal state and the external conditions before the most adaptive action is selected. The expected outcome is the driving force that shapes the final action (Heisenberg, 2014, 2015).

The plasticity of innate behaviors is commonly interpreted under the sensory-response model as the existence of a wider repertoire of hard-wired innate routines, each being triggered by a combination of external stimuli and internal drivers. At least two possible scenarios exist under this view. In the first one, every routine has its own neuronal substrate. Any given situation will be considered a new input, activating a specific network that should inhibit all the other routines and behaviors to promote the most adaptive behavioral output. This is the classical view of innate behaviors as hard-wired. In the second scenario, there are fewer neuronal substrates, and these are not dedicated each to a specific routine but rather define the principal features of the behavior. Other modulatory inputs refine the behavioral output of the neuronal network, resulting in a broad spectrum of routines for a behavior. It is possible that both scenarios coexist. The first one could be possible for very simple behaviors with relatively little variation in their inputs. The alternative scenario represents one of the possibilities of how the internal state or the analyses of the internal state vs. external factors modulate a behavior. We do not yet fully understand how internal and external stimuli are integrated, nor how the networks that integrate those factors interact with the networks that trigger behaviors. We also do not know what each type of decision looks like at the neuronal level. It is always a possibility that what we perceive at an observational level as the activation of one of two possible mutually exclusive behaviors shares a lot at the neuronal level with what we describe as a complex decision. It is in this gap in understanding that I believe we could start thinking about whether and how cognitive processes could shape the final action in some innate behaviors.

Drosophila is an excellent model organism for this type of study. Their small size, their great repertoire of behaviors, and the availability of advanced genetic tools (reviewed in Owald et al., 2015; Luo et al., 2018) give us the ability to address these questions at the level of behavior, circuits, and individual cells. Current technology allows us to specifically and reversibly manipulate one or several neurons in living, behaving flies. This makes it possible to dissect the circuits dedicated to behavioral flexibility, decisions, and cognitive processes, and see how different, or not, they are at the neuronal level, and how common they are for different behavioral choices.

Menzel et al. (2007) defined cognition as the "use and handling of knowledge, which allows the animal to decide between different options in reference to the expected outcome of its potential actions" and provided three essential characteristics of cognition as part of the cognitive components of behavior: (1) rich and cross-linked forms of sensory and motor processing; (2) flexibility and experience-dependent plasticity in choice performance; and (3) long-term adaptation of behavioral routines. The goals of this opinion article are to highlight the already known but underestimated complexity of innate behaviors and to explicitly associate these studies with the concept of cognition. Although none of the following examples fall strictly under the definition of cognition, certain aspects of the processes leading to the modulation of these behaviors are similar to the cognitive components of behavior listed above.

In Drosophila, the giant fibers originate in the brain, and project down contralaterally to motor neurons that control the musculature responsible for jump–flight behaviors (reviewed in Allen et al., 2006). A single spike in these neurons is normally sufficient to cause a fly to take-off, resembling a visually-evoked escape response. Consequently, giant fibers were considered command neurons for these behaviors and escape responses as reflexes. However, research conducted over the last 10 years indicates that these responses are more elaborated, extending beyond the Giant Fiber motor outputs (Card, 2012). Drosophila escape behavior contains a sequence of at least three maneuvers (freezing, body leaning or leg posture adjustment, and wing elevation) that end with a jump, but with some degree of independence between each maneuver, allowing the fly to stop the sequence if it chooses to Card and Dickinson (2008a,b) and Card (2012). Each step comprises the addition of new information, resulting in a more variable and carefully shaped final action. This means that even in the small temporal window before a predator reaches the fly, the insect must select from a wide range of evasive maneuvers. Recently, it has been shown that looming stimuli (possibly indicating the approach of an attacker) produce a bimodal distribution in Drosophila escape response—with either short or long take-offs—that can be biased toward short take-offs by increasing stimulus speed (von Reyn et al., 2014). giant fibers are necessary and sufficient for short maneuvers, while long maneuvers require a parallel pathway. Linear integration of angular velocity and angular size from looming stimuli takes place in giant fibers and derives in action selection (von Reyn et al., 2017). Hence, adult Drosophila escape responses involve more neural control elements than a single command neuron, allowing a variety of computational and decision steps to take place before the evasive behavior occurs. Another common defensive strategy is freezing, where the animal remains still, reducing its chances of being noticed. A new study showed that Drosophila flies adopt a freezing strategy in a state-dependent manner (Zacarias et al., 2018). For this study, the authors developed a different setup from the one used in the escape-behavior studies previously mentioned. Flies faced 20 repeated inescapable looming stimuli instead of a single escapable looming stimulus. Under this condition, flies rarely jumped in response to the stimulus, and most of them froze. Even the flies that initially jumped ended up modifying their defensive strategy during the experiment, since the probability of jumping decreased over the course of the stimulus presentations, and the proportion of flies freezing increased. The decision between fleeing and freezing was modulated by walking speed. If flies were grooming or moving slowly at the time of threat, they were more likely to adopt a freezing strategy. In this study, the authors also started to describe part of the network involved in freezing. Zacarias et al. (2018) perfectly illustrates

how experimental conditions can promote different behavioral outputs, and how the state of the animal shapes the final action. Far from stereotypic and automatic reactions, defensive behaviors appear to be carefully calculated. In the presence of a threat, the animal begins a cost–benefit computation (e.g., to eat or to adopt a defensive strategy). If the threat is near and inescapable, the fly will freeze or flee depending on the action that was performing at the time. If it is an escapable threat, then a visually mediated motor planning will determine the direction of the escape. Next, if the fly decides to jump, at least two types of take-offs could be performed, a short one in which speed is favored over wing stability or a long one that produces a steady flight.

Perhaps even more interesting is how the presence of parasitoid wasps affects oviposition in adult flies, as a mechanism to protect their offspring from a possible future threat. Parasitoid wasps are not dangerous to adult Drosophila, but upon encountering female wasps, female flies adopt different strategies that include choosing food containing toxic levels of alcohol to lay their eggs (promoting the death of wasps' eggs and larvae; Kacsoh et al., 2013), and reducing oviposition rates (Lefevre et al., 2011). These behavioral switches rely on sight to sense wasps. Remarkably, the external conditions are assessed in terms of the danger they represent to their offspring and not to adult flies themselves. Similarly, by choosing food with elevated levels of ethanol, the probability of the fly's offspring being parasitized decreases, and at the same time if parasitization occurs, the fly larvae are more likely to survive (Milan et al., 2012). However, there is no instant benefit for the adults that chooses the substrate. Interestingly, neuropeptide F (NPF) and its receptor NPFR1 are involved in wasp-induced ethanol oviposition preference. NPF and NPFR1 are required for alcohol sensitivity (Wen et al., 2005), but they are also involved in the representation of the internal motivational states of hunger and satiety in the mushroom bodies via dopaminergic neurons that innervate the structure (Krashes et al., 2009). Given the preference-switch between normal food and ethanol-enriched food, and the known role of dopamine (DA) in value-based and goal-directed decision-making (Zhang et al., 2007; Schultz, 2010; Liu et al., 2012; Waddell, 2013), it would be worth investigating whether dopaminergic neurons are also recruited in this case.

Interestingly, flies form a nonassociative long-term memory of the exposure and will lay fewer eggs or choose alcoholenriched food to lay their eggs for 24–48 h after wasp exposure (Kacsoh et al., 2013, 2015). Strikingly, it has been shown that flies visually exposed to wasps can transmit oviposition reduction behavior to naive flies (Kacsoh et al., 2015), an interesting case of social learning (Grüter and Leadbeater, 2014). That is to say, flies that never encounter a wasp can acquire and use the knowledge of others to modify their oviposition behavior. Kacsoh et al. (2015) showed that oviposition reduction behavior of naive flies (students) could last for 24 h after they were separated from wasp-exposed flies (teacher), but they could not teach others. They also demonstrated that learning mutants were unable to teach or be students but showed normal acute oviposition reduction during wasp exposure. Visual cues alone are sufficient for acute reduction in oviposition and memory formation in teachers, and social-learning responses. However, social learning requires teachers to have intact wings for students to learn, suggesting a role for both wings in communication through visual cues (Kacsoh et al., 2015). It is also noteworthy that all these learning processes require the mushroom bodies, structures previously demonstrated to be important for valence and memory-based action selection (Zhang et al., 2007; Aso et al., 2014) and to contain and receive inputs from dopaminergic and octopaminergic neurons (Zhang et al., 2007; Kim et al., 2013; Waddell, 2013; Wu et al., 2013). DA and octopamine (OA) are key modulators of behavior. OA has been implicated in statedependent changes in visual processing (Longden and Krapp, 2009; Suver et al., 2012), experience-dependent modulation of aggression (Bonini, 2000; Stevenson et al., 2005; Hoyer et al., 2008), social decision-making (Certel et al., 2010), and reward (Burke et al., 2012). DA is also known for its roles in reward (Barron et al., 2010; Burke et al., 2012), motivation (Krashes et al., 2009; Zhang et al., 2016) and, as previously mentioned, value-based or goal-directed decision-making (Zhang et al., 2007; Liu et al., 2012; Waddell, 2013; Beeler et al., 2014). Both seem to be involved in mediating certain aspects of value albeit in different modalities or domains (Aso et al., 2010; Burke et al., 2012; Scheiner et al., 2014; Huetteroth et al., 2015).

Curiously, these two biogenic amines differently modulate phototaxis, in what it seems to be a goal-directed or value-based decision-making process. Phototaxis seems to be a special case of photopreference and manipulating the ability of flies to fly can reversibly shift it from approach to avoidance in walking flies (Gorostiza et al., 2016). Photopreference can be influenced by the shape, form, or degree of intactness of the wings, the ability of flies to move them, and the state of sensory organs related to flight. Hence, flies appear to constantly monitor their flying ability, even while walking as these experiments suggest, and adjust their photopreference accordingly. It is worth noting that the neuronal activity of dopaminergic and octopaminergic circuits is indispensable and inducing for the modulation of phototactic behavior, but with opposite effects, suggesting a potential role of DA and OA, and supporting the idea of a value-based decision-making process taking place. In this view, phototaxis is not a response, but an action selected only in rather particular circumstances after a central decision-making stage that negotiates external stimuli as well as internal demands. When flying ability is compromised, the value of the different consequences of moving toward light changes and the dangers become more prominent due to the difficulties to escape; hence, the flies choose to hide until the danger goes away or flying ability is restored. Immediately after emerging from the pupal case, all flies experience a flightless period during the wing expansion phase. In line with the results above, flies go through a phase of reduced phototaxis that extends beyond wing expansion until the stage when its wings render it capable of flying (Chiang, 1963). The alteration in flying ability may promote a shift in the expected outcome (Heisenberg, 2014, 2015), which would eventually drive the selection of an alternative, more adaptive action, as seen in preference suppression assays where air, light, and gravitaxis cues were paired with aversive stimuli (Seugnet et al., 2009; Baggett et al., 2018). In those cases, flies learn that cues that usually indicate an escape route will lead them to negative outcomes (an aversive taste or an aversive temperature). Noticeably, wing

expansion in flies also involves a decision process. After emerging from the pupal case, flies select a suitable perch and expand their wings, but wing expansion can be delayed under adverse environmental conditions, e.g., space restriction (Cottrell, 2009; Peabody et al., 2009). Work in Drosophila uncovered part of the neuronal network involved in the decision to expand the wings, and showed the connection with the decision to perch, which required an assessment of the external factors (Peabody et al., 2009).

These examples serve to demonstrate how innate behaviors can in fact be the outcomes of complex modulatory processes, careful assessment of factors and decisions, and not mere stereotypic and automatic responses. Through these examples, we can see some aspects that resemble cognitive components (Menzel et al., 2007): rich sensory and motor processing (escape response), experience-dependent plasticity in choice performance (oviposition), and flexible and long-term adaptation of behavioral routines (photopreference). In light of this, I argue that the way we frame and refer to these behaviors must change. We should think about them as behaviors dominated by innate components or preferences. This simple paraphrase can change the focus of the innateness from the behavior to some component of it, moving also the preprogrammed conception with it. As mentioned by Menzel et al. (2007), these components or preferences "seem to be essentially useful in guiding the animals' behavior in their first confrontations with the external world." Innate preferences could be extremely relevant in the absence of contradictory cues. Nonetheless, they are certainly not the only things determining the final shape of the behavior. When other factors add complexity to the situation, the innate component becomes diluted and lose strength, leaving only the behavior "without its innateness." Under simple and controlled circumstances (a fly in a tube with a source of light at one end), the behavior looks like

#### REFERENCES


a stereotypic and automatic response (light is turned on, and in most cases the fly approaches the source of light). In this case, the innate component is the only relevant factor for the behavior. However, in complex situations (the flying ability of the fly is compromised), other factors become prominent and the innate component loses relevance or becomes maladaptive. The innate preference becomes another factor to be considered. I propose that in that complex situation, a cognitive process is engaged in the final tuning of the behavior (the fly avoids the light). Hence, cognition could prevent automatic maladaptive responses and also help fine-tune "innate routines," depending on the combination of external stimuli and internal drivers. We should carefully consider the cognitive aspect of any behavior we study, no matter how seemingly dominated by innate components or stereotypic it looks.

#### AUTHOR CONTRIBUTIONS

The author confirms being the sole contributor of this work and approved it for publication.

# FUNDING

This article was supported by an IBRO Return Home Fellowship.

#### ACKNOWLEDGMENTS

Many thanks to Björn Brembs for extensive discussions on this topic. Thanks also to Tomer Czaczkes, Christian Rohrsen, Luciana Pujol-Lereis, and Lia Frenkel for critical reading of the manuscript. I am a member of the National Scientific and Technical Research Council (CONICET) and the National University of Cordoba (UNC). This work was financially supported by CONICET, and an IBRO Return Home Fellowship.


Drosophila using targeted expression of the TRPM8 channel. J. Neurosci. 29, 3343–3353. doi: 10.1523/JNEUROSCI.4241-08.2009


**Conflict of Interest Statement:** The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2018 Gorostiza. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Bumblebees Express Consistent, but Flexible, Speed-Accuracy Tactics Under Different Levels of Predation Threat

Mu-Yun Wang1,2, Lars Chittka1,3 and Thomas C. Ings1,4 \*

<sup>1</sup> Department of Biological and Experimental Psychology, School of Biological and Chemical Sciences, Queen Mary University of London, London, United Kingdom, <sup>2</sup> Graduate School of Arts and Sciences, The University of Tokyo, Tokyo, Japan, <sup>3</sup> Institute for Advanced Study, Berlin, Germany, <sup>4</sup> Department of Biology, Anglia Ruskin University, Cambridge, United Kingdom

#### Edited by:

Patrizia d'Ettorre, Université Paris 13, France

#### Reviewed by:

Nicolas Chaline, Universidade de São Paulo, Brazil Dominic Standage, Queen's University, Canada

> \*Correspondence: Thomas C. Ings thomas.ings@anglia.ac.uk

#### Specialty section:

This article was submitted to Comparative Psychology, a section of the journal Frontiers in Psychology

Received: 27 February 2018 Accepted: 13 August 2018 Published: 03 September 2018

#### Citation:

Wang M-Y, Chittka L and Ings TC (2018) Bumblebees Express Consistent, but Flexible, Speed-Accuracy Tactics Under Different Levels of Predation Threat. Front. Psychol. 9:1601. doi: 10.3389/fpsyg.2018.01601 A speed-accuracy trade-off (SAT) in behavioural decisions is known to occur in a wide range of vertebrate and invertebrate taxa. Accurate decisions often take longer for a given condition, while fast decisions can be inaccurate in some tasks. Speed-accuracy tactics are known to vary consistently among individuals, and show a degree of flexibility during colour discrimination tasks in bees. Such individual flexibility in speed-accuracy tactics is likely to be advantageous for animals exposed to fluctuating environments, such as changes in predation threat. We therefore test whether individual speedaccuracy tactics are fixed or flexible under different levels of predation threat in a model invertebrate, the bumblebee Bombus terrestris. The flexibility of speed-accuracy tactics in a foraging context was tested in the laboratory using a "meadow" of artificial flowers harbouring "robotic" crab spider predators. We found that while the ranking of bees along the speed and accuracy continuums was consistent across two levels of predation threat, there was some flexibility in the tactics used by individual bees – most bees became less accurate at colour discrimination when exposed to predation threat when flower types were rewarding. The relationship between decision speed and accuracy was influenced by predator detectability and the risk associated with making incorrect choices during the colour discrimination task. Predator crypsis resulted in a breakdown in the relationship between speed and accuracy, especially when making an incorrect floral choice incurred a distasteful quinine punishment. No single speed-accuracy tactic was found to be optimal in terms of foraging efficiency under either predation threat situation. However, bees that made faster decisions achieved higher nectar collection rates in predator free situations, while accurate bees achieved higher foraging rates under predation threat. Our findings show that while individual bees remain relatively consistent in terms of whether they place greater emphasis on speed or accuracy under predation threat, they can respond flexibly to the additional time costs of detecting predators.

Keywords: animal personality, Bombus terrestris, predation risk, predator crypsis, speed-accuracy trade-offs

# INTRODUCTION

fpsyg-09-01601 August 31, 2018 Time: 19:4 # 2

Choices made by animals frequently involve a trade-off between decision speed and decision accuracy (Wickelgren, 1977; Chittka et al., 2009; Heitz and Schall, 2012), with fast decisions tending to be less accurate than slow decisions for a given task condition. A speed-accuracy trade-off (SAT) has been shown to occur during discrimination tasks across a wide range of taxa including humans (Simen et al., 2009; Bogacz et al., 2010), non-human primates (Heitz and Schall, 2012), birds (Ducatez et al., 2015), fish (Wang et al., 2015), and insects such as bees (Chittka et al., 2003; Burns and Dyer, 2008). The majority of studies on SAT, especially in humans and non-human primates, have used the SAT as a paradigm for exploring behavioural flexibility in decision making and choice behaviour (e.g., Fitts, 1966; Wickelgren, 1977) and its neuronal basis (reviewed by Standage et al., 2014 and also see Heitz and Schall, 2012; Hanks et al., 2014 for nonhuman primate examples). More recently, researchers working on animals, including bees, birds, and fish (Chittka et al., 2003; Ducatez et al., 2015; Wang et al., 2015; but also see Phillips and Rabbit, 1995 for a human example), have also considered SAT from a different perspective, i.e., whether the SAT is a stable trait in which the individual differences are maintained within a population over time.

Response speed has long been an important component describing individually consistent behavioural traits in vertebrates, such as shyness-boldness or neophobia (Van Oers et al., 2005; Toms et al., 2010). These traits, related to response speed, can be heritable (Drent et al., 2003) and different traits can be adaptive depending upon environmental changes (Dingemanse et al., 2004). In invertebrates, this approach has recently been used to consider the relationship between decision speed and accuracy. For example, Chittka et al. (2003) showed that foraging bumblebees express inter-individual variation in speed-accuracy tactics during floral colour discrimination. This variation remained consistent even when the cost of making errors increased, although all bees became slower and more accurate. Similar individual variation in speed-accuracy tactics has also been shown in honeybees (Burns and Dyer, 2008). While these studies indicate that speed-accuracy tactics in invertebrates do vary consistently among individuals, and that there is some flexibility at the level of the individual (Chittka et al., 2003), we still have limited understanding of how they can be adjusted to match changing situations (Chittka et al., 2009), such as increased predation threat.

Levels of inter-individual variability of behavioural phenotypes (O'Steen et al., 2002; Dingemanse et al., 2004) and individual behavioural flexibility (Herborn et al., 2014) are believed to be influenced by environmental fluctuation. We would therefore expect selection to favour flexibility in speed-accuracy tactics in social animals, such as bumblebees, adapted to dynamic environments (reviewed in Klein et al., 2017) where factors such as food availability and predation risk vary temporally and spatially. Bumblebees are social insects where the worker caste collects food (nectar and pollen from flowering plants) for the entire colony (Goulson, 2003). Foraging bees maximise their foraging efficiency by processing visual and olfactory cues to select the flowers of plants that provide the greatest returns (Chittka et al., 1999; Chittka and Raine, 2006). However, the best available options vary considerably through time and space (Von Buttel-Reepen, 1900; Arnold et al., 2009), and foraging bees need to avoid predators such as the crab spider Misumena vatia which hunts on flowers (Morse, 2007). The risk of predation from these predators also varies from patch to patch (Morse, 2007) and, due to their ability to change colour, the detectability of the spider can vary depending upon the colour of the flower it is hunting on (Chittka, 2001).

We therefore use a model invertebrate, the bumblebee Bombus terrestris, and an established predator avoidance learning paradigm (Ings and Chittka, 2008, 2009), to examine the flexibility of individual speed-accuracy tactics in response to changing predation risk. Bees are exposed to a natural scenario where they have to discriminate between two similar flower types to maximise energy intake in an artificial meadow where the risk of predation by model crab spiders is added half way through the experiment. In this design, introduction of predation risk changes the decision task slightly. We therefore do not focus on classical SAT, where participants perform the same task in different motivational conditions (speed or accuracy emphasis), but rather, we focus on inter-individual variation and intraindividual consistency in decision speed and accuracy under changing predation risk. Our main questions are: (1) Do bees maintain consistent speed-accuracy tactics in a floral colour discrimination task when exposed to increased predation risk? (2) Does the optimal speed-accuracy tactic change with predation risk and the difficulty of detecting predators? We hypothesise that the speed-accuracy tactic employed by individual bees will be flexible and that the optimal speed-accuracy tactic will differ depending upon predator crypsis and costs of incorrect choices in flower colour discrimination.

# MATERIALS AND METHODS

#### Study Animals

Bumblebees (B. terrestris dalmatinus, Dalla Torre 1882) from three colonies obtained from a commercial supplier (Syngenta Bioline Bees, Weert, Netherlands) were used in the experiment. Individual bees were marked with numbered tags (Christian Graze KG, Weinstadt-Endersbach, Germany). All colonies were maintained at room temperature (23◦C) and exposed to a 12:12 h light/dark cycle, with the light phase starting at 8 am. All colonies were supplied with ad libitum sucrose solution (50%, v/v) and pollen.

# Experimental Apparatus

Full details of the experimental apparatus are provided in Ings and Chittka (2008) and Ings and Chittka (2009). This experiment was carried out in a wooden flight arena (l = 1 m, w = 0.72 m, and h = 0.73 m) with a UV-transmitting Plexiglas lid. Controlled lighting was provided by two twin lamps [TMS 24 F with HF-B 236 TLD (4.3 kHz) ballasts, Philips, Netherlands], fitted with Activa daylight fluorescent tubes (Osram, Germany), which were suspended above the flight arena. A four by four vertical

array of artificial flowers (7 cm × 7 cm flat cards painted with yellow acrylic colours) was presented on a grey background on the end wall of the arena (**Figure 1A**). Bees entered the arena through an entrance tunnel attached to the opposite wall to the meadow. Each artificial flower (**Figure 1B**) consisted of a small wooden landing platform (40 mm × 60 mm), 10 mm under a small hole through which bees could access rewards (sucrose solution). Syringe pumps (KD Scientific, KD200, Holliston, MA, United States) were used to provide a continuous supply of sucrose solution at the tips of 26G syringe needles (BD Microlance Drogheda, Ireland; 0.45 mm × 13 mm) placed behind the access hole on the flowers.

To simulate predation risk, robotic "spider arms" (custombuilt by Liversidge & Atkinson, Romford, United Kingdom) covered with sponges (**Figures 1A,B**) were set up at the base of the flowers to simulate predation attempts (detailed in Ings and Chittka, 2008, 2009). To provide realistic visual predator cues, a 12 mm wide three dimensional model (made from Gedeo Crystal resin) of a crab spider (M. vatia) was placed just above the feeding hole on the "dangerous flowers" (**Figure 1B**). Full details of the dangerous flowers, including spectral reflectance of the background, spiders and flowers can be found in Ings and Chittka (2008) and Wang et al. (2013).

### Pre-training

To allow bees to become accustomed to the arena and flowers, all bees were given unrestricted access to the flight arena for a minimum of 1 day prior to the beginning of the experiments. No floral signals were placed in the artificial meadow to avoid bees developing any colour bias prior to the experiments. However, all flowers were supplied with a constant flow (1.85 ± 0.3 µl per minute) of 50% (v/v) sucrose. Individual bees that had continued feeding for a minimum of three foraging bouts (i.e., they entered the arena, collected sucrose solution from the artificial flowers and returned to the nest on at least three consecutive occasions) were used in the experiments.

#### Experimental Design

Full details of the experimental procedure are provided in Wang et al. (2013) and summaried in **Figure 2**. During the training phase, the 7 cm × 7 cm floral signals were added to the artificial meadow (**Figure 1A**). Bees were then trained to distinguish between two similar shades of yellow artificial flowers (for details of the colours see Wang et al., 2013) for 200 flower choices - a bee needed to land on the platform of a flower and probe for artificial nectar to be deemed a choice. In two groups (conspicuous spider and cryptic spider), the dark yellow flowers were more rewarding [50% (v/v) sucrose solution] than the light yellow flowers [20% (v/v) sucrose solution]. To encourage flower discrimination in a third group (quinine and cryptic spider – hereafter referred to as just 'quinine'), we replaced the 20% (v/v) sucrose solution with a distasteful 0.12% (w/v) quinine hemisulfate solution that bees are known to rapidly learn to avoid (Chittka et al., 2003). In the testing phase, we introduced spider models, either highly conspicuous (white; conspicuous spider group) or cryptic (same colour as the flowers; cryptic spider and quinine groups) to two of the eight (i.e., 25%) high quality (dark yellow) flowers and

tested the bees for another 200 choices. Three to six foraging bouts were required for bees to make 200 choices, and bees were allowed to complete their final foraging bout and return to the nest (see Wang et al., 2013 for further details). When bees landed on "dangerous" flowers (with a spider model), they were immediately exposed to a simulated predation attempt by being held by the arms of a "robotic crab spider" for 2 s – thus they had no opportunity to collect the sucrose solution. The positions of the flowers were changed in a pseudo-random fashion between each foraging bout (at least three were required to attain 200 choices): positions of different flower types were changed randomly, but the number of high reward flowers in both the top and bottom two rows of the meadow were maintained the same in order to avoid spatial preference bias.

sponge coated pincers for 2 s when a bee lands to feed). It also shows the

syringe tip which dispenses the sucrose or quinine solutions.

#### Data Analysis

The movements and positions of 44 bees from across the three colonies were recorded in real time during the experiment. Of

these, four were excluded from the analyses as they stopped foraging during the experiment (this left 15 bees in each of the conspicuous spider and cryptic spider groups and 10 bees in the quinine group). Three-dimensional coordinates of bee positions were calculated 50 times per second using two video cameras connected to a computer running Trackit 3D software (BIOBSERVE GmbH, Bonn, Germany). We calculated the time bees spent in the investigation zones, which were 7 cm (length) by 9 cm (width) by 9 cm (height) from the holes providing sucrose or quinine solution. Investigation zones were set based on the visual angles of bumblebees where bees were able to detect both flower signals and predators using colour contrast (Spaethe et al., 2001).

To remove learning effects, we calculated the colour discrimination accuracy (proportion of high reward flowers chosen) and decision speed during the final 30 choices (out of 200) made during each phase of the experiment. In the testing phase, only visits to high reward flowers without spiders were scored as correct choices. We used the average time spent inspecting flowers (duration in the investigation zones), rather than average time between choices (e.g., as in Chittka et al., 2003), as our measure of decision speed. Time between flowers is only a proxy of decision time and is influenced by other factors such as flight speed and path length between flowers. Prior to analysis, inspection time was converted to relative decision speed using the following formula: Speed = 1 – [(decision time – minimum decision time)/(maximum – minimum decision time)]. Thus the bee that took the longest (1.31 s per flower) to inspect flowers was scored as 0 and the fastest bee (0.31 s per flower) was scored as 1 (mean ± 1SE decision time = 0.60 ± 0.2 s per flower). All statistical analyses described below were carried out in RStudio 1.1.423 (R Core Team, 2015) running R 3.4.3 (R Core Team, 2017).

# The Relationship Between Speed and Accuracy

Before examining the consistency of floral colour discrimination speed-accuracy tactics across situations (change in predation risk), we used linear correlation analysis (Pearson's product moment correlation) to determine if speed and accuracy were related. Correlation was used as there was no a priori reason to expect decision speed to be dependent upon accuracy or vice versa. Each experimental group was examined separately. Normality tests (Shapiro–Wilk) and visual inspection of the scatterplots were undertaken to check that the assumptions of linear correlation were not violated.

# Consistency of Speed-Accuracy Tactics Under Different Levels of Predation Risk

To test whether floral colour discrimination speed-accuracy tactics are rigid or flexible with changing predation risk, we used a combination of linear regression and paired t-tests. While both speed and accuracy are proportions, inspection of the residuals from fitted models, along with normality tests (Shapiro– Wilk tests) showed that it was not necessary to transform these data or use generalised linear models. Linear regression was used to identify whether speed and accuracy during the testing phase were dependent upon speed and accuracy during the training phase (each group was analysed separately). The associated R 2 -values provide an index of the stability of the rank position of each bee in terms of speed or accuracy, i.e., whether the fastest bee remains the fastest bee under predation risk. The t-tests provide a measure of the consistency of the magnitude of speed and accuracy for individual bees within each group. Thus, if bees in a group maintained consistent speed-accuracy tactics under predation threat they would have high R 2 -values but low t-values. For the regression analysis, residual versus fitted value and quantile–quantile plots were inspected to check that each model met the assumptions of linear regression. The "linearHypothesis" function in the R package "car" version 2.14 (Fox and Weisberg, 2011) was used to test whether slope coefficients differed significantly from 1 (all bees remained equally consistent). The slope coefficient values were used to indicate effect sizes of the regressions and Hedges g was calculated in version 0.7.1 of the "effsize" package (Torchiano, 2017) in R to assess effect sizes for the mean differences in speed and accuracy between training and testing phases.

# Optimal Tactic Under Different Levels of Predation Risk

In a predation free environment, the optimal speed-accuracy tactic should yield the highest nectar collection rates. However, in an environment with a high predation risk, the optimal tactic will involve a trade-off in terms of nectar collection rate and avoiding being killed by predators. However, because all bees had a strong predator avoidance response by the end of training (Wang et al., 2013), there was insufficient variation to allow a meaningful analysis of the influence of speed-accuracy tactics on predation risk. Thus, we focused on nectar collection rates as our measure of optimality in relation to speed-accuracy tactics.

Nectar collection rate (mg sucrose per second) was calculated by dividing the amount of sucrose collected by each individual bee by the total time they spent foraging in the arena. Each of the high reward flowers provided approximately 4 mg of sucrose (4.7 µl of 50% v/v sucrose solution) while each low reward flower provided approximately 0.8 mg of sucrose (4.7 µl of 20% v/v sucrose solution). Therefore, during training there was on average 80% more sugar reward available in the high reward flowers compared to the low reward flowers. During testing, this difference reduced slightly to 73.3% as no sugar could be collected from dangerous flowers. Flowers containing quinine solution provided no sucrose reward, and when bees visited a flower harbouring a crab spider model they were captured before they could collect any sucrose solution.

First, we examined the relationship between decision time and total foraging duration using Pearson's product moment correlation. To meet the assumption of normality, total foraging duration was log (natural) transformed prior to the analysis. The influence of decision speed and accuracy on nectar collection rate was examined using separate general linear models for training and testing phases. Full models (including interactions) with experimental group and both speed and accuracy were initially fitted to determine which variable explained the greatest amount of variation. We included experimental group in the model to test whether the intercepts or slopes differed among the experimental treatments. After fitting the full model (including the interactions between group and speed and accuracy) we used the "Anova" function in the R package "car" to calculate Type III sums of squares to allow us to choose which variables to drop to improve the fit of the models. Model terms were dropped sequentially until we arrived at the minimum adequate model with the lowest Akaike Information Criterion (AIC). Residual versus fitted value plots, quantile–quantile plots and variance inflation factors (VIF) were inspected to check for any violations of model assumptions. We used the same procedure to test whether nectar collection rate during testing was dependent upon nectar foraging rate during training. The "linearHypothesis" function in the R package "car" was used to test whether slope coefficients differed significantly from 1 (all bees remained equally consistent).

# RESULTS

# The Relationship Between Speed and Accuracy

Decision speed (inverse of decision time) and accuracy were generally negatively correlated during both training and testing, i.e., some bees were slow and accurate, while others were fast and error prone. However, the degree of correlation differed between treatment groups (**Figures 3A–C**). In the absence of predation risk, speed and accuracy were significantly correlated in the cryptic spider group (Pearson's r<sup>13</sup> = −0.619, P = 0.013) but not the conspicuous spider group (Pearson's r<sup>13</sup> = −0.399, P = 0.141) or the quinine group (Pearson's r<sup>8</sup> = −0.008, P = 0.982). When the conspicuous spider and cryptic spider groups, which experienced identical conditions during training, were pooled, the overall relationship between speed and accuracy was strongly negatively correlated (Pearson's r<sup>28</sup> = −0.525, P = 0.003). Under predation risk, speed and accuracy were negatively correlated in the conspicuous spider group (Pearson's r<sup>13</sup> = −0.677, P = 0.006) but not the cryptic spider (Pearson's r<sup>13</sup> = −0.457, P = 0.086) or quinine groups (Pearson's r<sup>8</sup> = 0.317, P = 0.373).

# Consistency of Speed-Accuracy Tactics Across Situations

When bees moved from a single visual discrimination task to simultaneous colour discrimination and predator avoidance (testing phase), the individual consistency in both decision speed

and accuracy differed among experimental groups (**Figure 4** and **Table 1**). In the conspicuous spider group, both decision speed and accuracy during testing were dependent upon speed and accuracy during training (**Figure 4A** and **Table 1**), i.e., the rank position of individual bees remained consistent. Although there was a small (2% increase; Hedge's g estimate = −0.131, 95% CI = −0.879 to 0.618), but significant, increase in decision speed within the group (**Table 1**), this change was consistent for all individuals (linear regression: β = 1.065 ± 0.292; contrast against a slope of 1: F<sup>1</sup> = 0.05, P = 0.828). Individual accuracy within the group also fell slightly during testing (6% decrease; **Table 1**; Hedge's g estimate = 0.601, 95% CI = −0.163 to 1.366) and this change was consistent within the group (linear regression: β = 1.481 ± 0.275; contrast against a slope of 1: F<sup>1</sup> = 3.06, P = 0.104). In the cryptic spider group, the rank position of individual bees also remained constant for both speed and accuracy (**Figure 4B** and **Table 1**). Individual decision speed did not change significantly between phases (**Table 1**), but decision accuracy was strongly (by 23.6%) reduced (Hedge's g estimate = 2.421, 95% CI = 1.136 to 3.405) during the testing phase. Furthermore, the reduction in accuracy was greater for the bees which were most accurate in training (**Figure 4B**; linear regression: β = 0.584 ± 0.176; contrast against a slope of 1: F<sup>1</sup> = 5.62, P = 0.034). In contrast to the conspicuous spider and cryptic spider groups, there was no consistency in rank position, or changes in speed or accuracy between training and testing for bees in the quinine group (**Table 1**).

correlation coefficient for each series is indicated with stars (∗P ≤ 0.05, ∗∗P ≤ 0.01, NS for P > 0.05).

#### Optimal Tactic Under Different Levels of Predation Risk

The time taken to visit 200 flowers (natural log) was negatively correlated with relative decision speed during both training (Pearson's r<sup>38</sup> = −0.570, P < 0.001) and testing (Pearson's r<sup>38</sup> = −0.483, P = 0.002) phases. The nectar collection rate (foraging efficiency) during training was dependent upon decision speed (**Figure 5A**; linear regression: R <sup>2</sup> = 0.481, F1,<sup>36</sup> = 11.06, P < 0.001) and differed between treatment groups (**Figure 5A**; linear model: F2,<sup>36</sup> = 9.64, P = 0.003). A bee with 10% higher relative decision speed had a 20% greater nectar collection rate (linear model: β = 0.434 ± 0.140). In contrast, there was no difference in nectar collection rates between groups during testing (linear model: F2,<sup>35</sup> = 1.08, P = 0.351), and nectar collection rates were dependent upon a linear combination of decision accuracy and decision speed (linear regression: R <sup>2</sup> = 0.318, F2,<sup>37</sup> = 8.63, P < 0.001). Decision accuracy explained twice as much variation (25%; β = 0.393 ± 0.102) in nectar collection rate as decision speed (12%; β = 0.266 ± 0.099).

The nectar collection rate under predation risk was dependent upon nectar collection rate during training (linear model: R <sup>2</sup> = 0.337, F1,<sup>38</sup> = 19.27, P < 0.001), irrespective of experimental group (**Figure 6**). The relationship was positive, with the ranking of individual bees being consistent between phases, although the nectar collection rate during testing did not consistently match that during training (**Figure 6**). While most bees had lower nectar collection rates during testing (linear regression: β = 0.513 ± 0.117; contrast against a slope of 1: F<sup>1</sup> = 17.35, P < 0.001) a few bees, especially those in the conspicuous spider group, had higher nectar collection rates during testing.

#### DISCUSSION

We demonstrated that while the inter-individual expression of speed-accuracy tactics remained consistent under increased

spider (A), cryptic spider (B), and quinine (C) groups. Fitted lines represent significant predicted values from linear regression analyses. R <sup>2</sup> and associated P-values are shown in Table 1. To aid interpretation, the dashed grey line represents a hypothetical 1:1 relationship between testing and training phases – deviations from this show a non-uniform change in speed or accuracy along the speed and accuracy continuum.



Significant results (P ≤ 0.05) are shown in bold.

predation threat, they were flexible across changing situations. However, the consistency of the speed-accuracy tactics, and degree of flexibility, were dependent upon predator detectability and the costs of making errors in the colour discrimination task. When predators were easily detected, the relationship between colour discrimination speed and accuracy was consistent when predation threat increased. In contrast, when predator detection was difficult, the relationship between decision speed and accuracy broke down, especially when errors in the floral colour discrimination task were punished with bitter quinine. Although, caution is required interpreting the results of the quinine group due to a smaller sample size and low variation in speed and accuracy among individuals. While we did detect limited flexibility in the speed-accuracy tactics employed by individual bees, there was no evidence to support our hypothesis that bees employ an optimal (in terms of nectar collection rate) speed-accuracy tactic to match the level of predation risk and detectability of the predators. Therefore, we suggest that while perceptual and cognitive constraints in bumblebees may limit the flexibility of speed-accuracy tactics employed by individual bees, a diversity of individually consistent behavioural traits at the colony level may be advantageous in environments with fluctuating predation risk (Muller and Chittka, 2008).

# Consistency of Speed-Accuracy Trade-Off Tactics Across Situations

Even though the ranking of individual speed-accuracy tactics remained consistent with increased predation threat, there were changes in the magnitude of both speed and accuracy that differed among groups. The observed consistency in ranking of bees matches a previous study where the cost of making errors in colour discrimination was increased by use of gustatory punishment (Chittka et al., 2003). In their study, Chittka et al. (2003) found that bees shifted toward the slower-accurate end of the speed-accuracy continuum when errors were punished. In contrast, in our study (which unlike the earlier study, involved predation risk), we found that bees became less accurate, with

values from a significant linear regression model (R <sup>2</sup> = 0.481, F1,<sup>36</sup> = 11.13, P < 0.001) with a common slope but different intercepts for each group (conspicuous spider = black dotted line; cryptic spider = dashed grey line; quinine = solid black line. (B) Individual nectar collection rates during the testing phase are shown against decision accuracy and the black solid line represents predicted values from the linear regression (nectar collection rate ∼decision accuracy + decision speed; R <sup>2</sup> = 0.185, F1,<sup>13</sup> = 8.63, P = 0.006 ) with values for speed set at the group mean value. (C) Nectar collection rates are plotted against decision speed and the black solid line represents predicted values from the same linear regression as in (B) but with values for accuracy set at the group mean value. In all cases, the regression lines are not extrapolated beyond observed values.

little change in decision speed, when they had to solve a difficult colour discrimination task under predation threat. The change in accuracy was only minimal when spiders were conspicuous and was coupled with a very small increase in decision speed. This most likely reflects continued improvement of predator detection after training. In the quinine treatment, which was similar to that used by Chittka et al. (2003), we found no change in either speed or accuracy under predation risk, although interindividual variation was low during both phases. The results from this group did, however, support the findings from Chittka et al. (2003) which showed increased accuracy with the addition of gustatory punishment. These observations lead us to ask two important questions. First, why did predator detectability and the cost of errors affect the relationship between decision speed and accuracy under predation threat? Second, why did bees become less accurate at choosing the most rewarding flowers when under predation threat?

To answer these questions it is worth considering how bees process the visual information relating to predation risk and food rewards. During training, bees needed to discriminate between similar coloured flowers to maximise their foraging returns. Thus, when a bee perceived a flower it should have processed the visual appearance of the flower and matched it with the level of reward it received at similar flowers (Dyer and Chittka, 2004; Dyer et al., 2011). However, when a bee was exposed to the same flowers, but under predation risk, it needed to assess both the risk associated with feeding from a particular flower as well as the difference in reward it may receive (Ings and Chittka, 2008, 2009; Wang et al., 2013; Nityananda and Chittka, 2015). Bees could either (1) scan for predators then process the floral colour, (2) process the floral colour and then scan for predators, (3) simultaneously process floral colour and scan for predators, (4) just scan for predators and visit any safe flower, or, (5) avoid the risky flower type once they have made the association between colour and predation risk.

While bumblebees are believed to use restricted parallellike search (Morawetz and Spaethe, 2012) they still process scenes sequentially using active vision (Nityananda et al., 2014). Furthermore, although discrimination of contrasting colours requires shorter integration times than highly similar colours (Nityananda et al., 2014), bumblebees use a colour independent search image for spiders (Ings et al., 2012), i.e., complex shape recognition, that would also require longer integration times (Nityananda et al., 2014). It is therefore unlikely that they simultaneously process similar floral colours and scan for predators, although they can solve both discrimination tasks concurrently if strongly incentivised (Wang et al., 2013). Bees might avoid the risky flower type when spiders were cryptic and all flowers were rewarding (Ings and Chittka, 2009), but not when the alternative flower type is distasteful. Once bees make the association between predation risk and the highly rewarding flower type, they would only need to discriminate floral colour to avoid predation and reduce overall decision time. While bees exposed to cryptic spiders when both flowers were rewarding did indeed reduce their accuracy, they still visited too many safe high rewarding flowers to reflect avoidance of the risky flower type. It is therefore more likely that reduced accuracy reflects avoidance of spiders first followed by less accurate decisions in the colour discrimination task. This interpretation is supported by the fact that individual bees did not spend more time overall

making decisions (decision speed did not change), even though increased inspection times are required to detect cryptic spiders (Ings et al., 2012). Thus, to be able to maintain overall decision times, bees will have had less time to choose between high and low reward flowers due to time lost searching for cryptic spiders. This tactic, i.e., avoiding spiders as the top priority, and then foraging from any safe flower irrespective of reward, should yield greater rewards than a tactic avoiding all high reward flowers. Further evidence for bees using this tactic is given by the fact that a similar pattern was seen for the group of bees exposed to conspicuous spiders, although the accuracy only dropped slightly, reflecting the shorter amount of additional time needed to detect the conspicuous compared to cryptic spiders (Ings et al., 2012).

worse during testing than poorer nectar foragers.

Due to the importance of inspection time in calculating SAT, it is worth considering how its measurement may have influenced the observations. It could be argued that our measurement of inspection time, i.e., duration within a 7 cm × 9 cm × 9 cm zone in front of the flowers, may not capture the full decision process, thus leading to overestimated speed. While this is possible, inspection time and overall time between choices was highly correlated. Recent work has also shown that bumblebees may use active vision to distinguish complex patterns (spider shape) and similar colours, and that this requires side-to-side scanning of the scene (Nityananda et al., 2014). Such scanning behaviour has already been demonstrated to occur within the decision zone used in our experimental paradigm, especially when spiders are cryptic (Ings et al., 2012). We are therefore confident that our measure of inspection time does indeed accurately represent decision speed.

### Optimal Tactic Under Different Levels of Predation Risk

Our results showed that overall foraging duration was related to decision speed, with faster foraging bouts corresponding with faster decisions (**Figure 5**). The importance of decision speed was further borne out by the observation that foraging efficiency (nectar collection rate) in predator free environments was positively related to decision speed, but not accuracy (**Figure 5A**). Even though the two similar coloured flower types yielded very different rewards, visiting more flowers per unit time yielded a greater foraging efficiency. In contrast, decision accuracy was more important in determining foraging efficiency under predation threat, although decision speed still had some influence on foraging efficiency (**Figures 5B,C**). These results partially support the theoretical study of Burns (2005), which, using data from Chittka et al. (2003), predicted that fast, inaccurate decisions would be optimal in predator free environments, but slow, accurate decisions would be optimal under predation threat. Furthermore, a slow, accurate tactic should be favoured under predation threat because the proportion of "safe" rewarding flowers was low (Burns and Dyer, 2008) compared to when there was no predation risk. An alternative explanation could relate to the importance of avoiding being killed by a predator, such that the optimal foraging tactic under predation threat is to avoid predators irrespective of the cost to foraging efficiency. Indeed bees do maintain high levels of predator avoidance accuracy, despite the costs of detecting predators (Ings and Chittka, 2008, 2009).

An important point to consider is that although fast decisions were better in the predator free environment, and accurate decisions were better under predation threat, individual bees did not shift their tactic sufficiently to match the level of predation risk. This is borne out by the observation that nectar collection rates under predation risk were lower than those during training, even though they were strongly related. This reflects the fact that shifting behaviour to match the optimal tactic for changing situations can be costly when acquisition of information is difficult (DeWitt et al., 1998), or when the environment changes rapidly (predation risk in our case). In such situations, colonies with a diversity of individual speed-accuracy tactics (Burns and Dyer, 2008; Pruitt and Riechert, 2011), analogous to bet-hedging genotypes (Seger, 1987), could, on average, perform better in environments with fluctuating predation risk.

#### CONCLUSION

Our study has shown that, bumblebees, which have evolved in fluctuating environments, show a degree of flexibility of speed-accuracy tactics in response to changing predation threat. However, no individual speed-accuracy tactic resulted in optimal foraging efficiency under different levels of predation threat. We

suggest that this reflects perceptual and cognitive constraints that limit the flexibility of tactics expressed by individual bees. One possibility is that, as for many other traits in social species (reviewed in Jandt et al., 2014), including behavioural traits (Muller and Chittka, 2008), the diversity of speed-accuracy tactics at the colony level may be more important than individual tactics that are optimal under set circumstances. However, further work is required to test this possibility.

#### DATA AVAILABILITY STATEMENT

The raw data supporting the conclusions of this manuscript will be made available by the authors, without undue reservation, to any qualified researcher.

#### ETHICS STATEMENT

Only commercially reared bumblebees were used in this study and our protocol conformed to the regulatory requirements for animal experimentation in the United Kingdom.

# AUTHOR CONTRIBUTIONS

TI and LC designed the predator avoidance paradigm. M-YW and LC designed the experiments with input from TI.

## REFERENCES


Experiments were carried out by M-YW and statistical analyses were conducted by M-YW and TI with input from LC. MY-W and TI prepared figures and wrote the manuscript with contributions from LC.

# FUNDING

The development of the experimental paradigm was supported by Natural Environment Research Council (Grant NE/D012813/1) awarded to LC and TI. M-YW was supported by the Overseas Research Student Awards Scheme and the Ministry of Education and National Science Council Taiwan Studying Abroad Scholarship. During writing, LC was supported by HFSP program grant (RGP0022/2014) and EPSRC program grant Brains-on-Board (EP/P006094/1).

### ACKNOWLEDGMENTS

We thank Syngenta Bioline Bees for providing bumblebee colonies. We would also like to thank Dr. Sophie Mowles for valuable discussions regarding consistent individual behaviour and helpful comments on an early draft of the manuscript. Our thanks also go to the three reviewers and the editor, Patrizia d'Ettorre, for their helpful comments on an earlier draft.



**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2018 Wang, Chittka and Ings. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Kenyon Cell Subtypes/Populations in the Honeybee Mushroom Bodies: Possible Function Based on Their Gene Expression Profiles, Differentiation, Possible Evolution, and Application of Genome Editing

#### Shota Suenami †,‡, Satoyo Oya‡ , Hiroki Kohno‡ and Takeo Kubo\*

Department of Biological Sciences, Graduate School of Science, The University of Tokyo, Bunkyo-ku, Tokyo, Japan

Mushroom bodies (MBs), a higher-order center in the honeybee brain, comprise some subtypes/populations of interneurons termed as Kenyon cells (KCs), which are distinguished by their cell body size and location in the MBs, as well as their gene expression profiles. Although the role of MBs in learning ability has been studied extensively in the honeybee, the roles of each KC subtype and their evolution in hymenopteran insects remain mostly unknown. This mini-review describes recent progress in the analysis of gene/protein expression profiles and possible functions of KC subtypes/populations in the honeybee. Especially, the discovery of novel KC subtypes/populations, the "middle-type KCs" and "KC population expressing FoxP," necessitated a redefinition of the KC subtype/population. Analysis of the effects of inhibiting gene function in a KC subtype-preferential manner revealed the function of the gene product as well as of the KC subtype where it is expressed. Genes expressed in a KC subtype/population-preferential manner can be used to trace the differentiation of KC subtypes during the honeybee ontogeny and the possible evolution of KC subtypes in hymenopteran insects. Current findings suggest that the three KC subtypes are unique characteristics to the aculeate hymenopteran insects. Finally, prospects regarding future application of genome editing for the study of KC subtype functions in the honeybee are described. Genes expressed in a KC subtype-preferential manner can be good candidate target genes for genome editing, because they are likely related to highly advanced brain functions and some of them are dispensable for normal development and sexual maturation in honeybees.

Keywords: honeybee, hymenoptera, brain, mushroom body, Kenyon cell, learning and memory, genome editing

The European honeybee (Apis mellifera L.) is a social insect (Winston, 1986; Seeley, 1995), and its colony members exhibit advanced learning abilities that can be relatively easily assayed using associative learning paradigms, even under laboratory conditions (Takada, 1961; Giurfa et al., 2001; Dyer et al., 2005; Hori et al., 2006, 2007). Therefore, the honeybee has long been used as a model animal for studying learning and memory in insects (Giurfa, 2007; Giurfa and Sandoz, 2012; Chittka, 2017).

#### Edited by:

Martin Giurfa, UMR5169 Centre de Recherches sur la Cognition Animale (CRCA), France

#### Reviewed by:

Axel Brockmann, National Centre for Biological Sciences, India Amelie Cabirol, University of Trento, Italy Jean-Marc Devaud, Université Toulouse III Paul Sabatier, France

> \*Correspondence: Takeo Kubo stkubo@bs.s.u-tokyo.ac.jp

#### †Present Address:

Shota Suenami, Bioproduction Research Institute, National Institute of Advanced Industrial Science and Technology, Tsukuba, Japan ‡These authors have contributed equally to this work

#### Specialty section:

This article was submitted to Comparative Psychology, a section of the journal Frontiers in Psychology

Received: 19 March 2018 Accepted: 24 August 2018 Published: 02 October 2018

#### Citation:

Suenami S, Oya S, Kohno H and Kubo T (2018) Kenyon Cell Subtypes/Populations in the Honeybee Mushroom Bodies: Possible Function Based on Their Gene Expression Profiles, Differentiation, Possible Evolution, and Application of Genome Editing. Front. Psychol. 9:1717. doi: 10.3389/fpsyg.2018.01717

Drafts of the honeybee whole genome sequence (Honeybee Genome Sequencing Consortium, 2006; Elsik et al., 2014) have greatly promoted studies of the honeybee molecular biology, neuroscience, and genetics. This mini-review focuses on a topic that has received little attention to date–the possible roles of KC subtypes that constitute the MBs, a higher-order center in the honeybee brain (Erber et al., 1980; Rybak and Menzel, 1998; Komischke et al., 2005; Locatelli et al., 2005; Menzel and Manz, 2005; Ito et al., 2008; Szyszka et al., 2008), and their possible evolution in hymenopteran insects.

## UNIQUE GENE/PROTEIN EXPRESSION PROFILES OF KC SUBTYPES IN THE HONEYBEE BRAIN

# KC Subtypes That Constitute the Honeybee Mushroom Bodies

Several combinations of approaches including behavioral, pharmacological, electrophysiological, imaging, and ablation studies have revealed that mushroom bodies (MBs) play important roles in learning and memory, and sensory integration in the honeybee (Erber et al., 1980; Rybak and Menzel, 1998; Komischke et al., 2005; Locatelli et al., 2005; Menzel and Manz, 2005; Ito et al., 2008; Szyszka et al., 2008). In the honeybee, the MBs are a paired structure, each of which has two cuplike structures, called calyces, that are sensory input regions of the MBs (**Figure 1A**).

Honeybee MBs have long been thought to comprise three classes/subtypes of interneurons termed Kenyon cells (KCs): class I "classical" large- (lKCs or inner noncompact KCs) and "classical" small-type KCs (sKCs or inner compact KCs), and class II KCs (or outer compact KCs), which are distinguished by their cell body size and location in the MBs (**Figure 1B**) (Mobbs, 1982; Strausfeld et al., 1998; Strausfeld, 2002; Farris et al., 2004; Farris, 2005; Fahrbach, 2006). The somata of "classical" class I lKCs are located at the inside edges of the MB calyces, whereas those of "classical" sKCs are located in the inner core of the MB calyces. The somata of class II KCs, on the contrary, are located at the outer surface of the MB calyces (**Figure 1B**) (Mobbs, 1982; Strausfeld et al., 1998; Strausfeld, 2002; Farris et al., 2004; Farris, 2005; Fahrbach, 2006). However, each of the "classical" lKCs projects its dendrites to the olfactory (lip) or visual (collar) subregions of the MB calyces, and the "classical" sKCs project their dendrites to the multimodal basal ring. Class II KCs project their dendrites to the entire calyx (Strausfeld, 2002; Farris et al., 2004).

Recently, Kaneko et al. (2013) identified the novel class I mKCs, which are characterized by the preferential expression of middle-type-Kenyon cell-preferential arrestin-related protein (mKast) (**Figure 1C**) (Kaneko et al., 2013). Therefore, the honeybee MBs actually comprise three subtypes of class I KCs: "redefined" lKCs, mKCs, and "redefined" sKCs. The somata of the mKCs are localized between the "redefined" lKCs and "redefined" sKCs, and the size of the somata of the mKCs is intermediate between the "redefined" lKCs and "redefined" sKCs (**Figure 1C**; Kaneko et al., 2013). Importantly, these KC subtypes exhibit

differential gene expression profiles, suggesting they have distinct cellular characteristics and functions.

## lKCs

Honeybee MBs express more than 20 genes in a lKC subtypepreferential manner (for more comprehensive reviews, see Kubo, 2012; Kaneko et al., 2016). Among these genes, nine are expressed preferentially in the lKCs. Five of these 9 genes encode proteins involved in the intracellular Ca2+-signaling pathway, such as Ca2+/calmodulin-dependent protein kinase II (CaMKII) (Kamikouchi et al., 1998, 2000; Sen Sarma et al., 2009; Uno et al., 2012), which has an important role in the synaptic plasticity that underlies learning and memory abilities in various animals (Colbran and Brown, 2004; Elgersma et al., 2004; Pasch et al., 2011). Furthermore, Pasch et al. (2011) reported that phosphorylated (activated) CaMKII protein (pCaMKII) is present in lKCs, but not in sKCs or class II KCs (Pasch et al., 2011). These findings suggest that the lKCs are related to Ca2+ signaling-based learning and memory functions (**Figure 1D**; Ghosh and Greenberg, 1995; Rose and Konnerth, 2001; Perisse et al., 2009; Shonesy et al., 2014).

Matsumoto et al. (2014) used pharmacologic inhibition to indicate that CaMKII is involved in late long-term memory (LTM), but not in mid-term memory (MTM) or early LTM formation (Matsumoto et al., 2014). In addition, Scholl et al. (2015) used RNA interference (RNAi) and pharmacologic inhibition to indicate that CaMKII is necessary for both early and late LTM, but not for MTM (Scholl et al., 2015). Although the two studies reported different effects of CaMKII inhibition on early LTM, they consistently suggest that the lKCs play a role at least in late LTM formation in the honeybee.

Genes encoding for two transcription factors, Mushroom body/large-type Kenyon cell-preferential gene-1 [(Mblk-1)/E93] (Takeuchi et al., 2001) and Broad-Complex (BR-C) (Paul et al., 2006), are also expressed preferentially in the lKCs in the honeybee MBs. The MBR-1, a nematode homolog of Mblk-1, is necessary for both pruning excessive neurites during development and learning ability (Kage et al., 2005; Hayashi et al., 2009). Thus, selective expression of Mbk-1 in the lKCs is consistent with the speculation that synaptic plasticity is enhanced in the lKCs. It is also plausible that Mblk-1 and BR-C are involved in transactivation of genes expressed in an lKCpreferential manner in the honeybee brain.

Suenami et al. (2016) recently identified three genes, synaptotagmin 14 (Syt14), discs large 5 (dlg5), and phospholipase C epsilon (PLCe), whose expression is more highly enriched in the MBs of the honeybee brain than the previously identified KC subtype-preferential genes (Suenami et al., 2016). While, Syt14 and dlg5 are highly selectively expressed in the "redefined" lKCs in the MBs, PLCe is highly expressed in the whole MBs; i.e., all of the class I lKCs, mKCs, and sKCs and class II KCs (**Figure 1D**; Suenami et al., 2016). Syt14 and dlg5 are involved in membrane trafficking and spine formation, respectively (Fukuda, 2003; Hayashi et al., 2009; Doi et al., 2011; Wang et al., 2014; Suenami et al., 2016), implying that both synaptic transmission and synaptic plasticity are enhanced in the lKCs.

It is difficult to conclude definitely on the correspondence between the "classical" lKCs and "classical" sKCs, and "redefined" lKCs, mKCs, and "redefined" sKCs based on morphological observation. In some previous studies, which reported on genes expressed in a lKC-preferential manner, it seems that "classical" lKCs correspond to "redefined" lKCs, and "classical" sKCs correspond to mKCs plus "redefined" sKCs [for example, (Kamikouchi et al., 2000; Takeuchi et al., 2001; Uno et al., 2012)]. On the contrary, Strausfeld (2002) previously represented the boundary between the "classical" lKCs and "classical' sKCs, which are distinguished based on their morphology, just in the mKC area (Strausfeld, 2002). Therefore, future studies must investigate the actual correspondence between the "classical" lKCs and "classical" sKCs, and the "redefined" lKCs, mKCs, and "redefined" sKCs, by examining their gene expression profiles using double in situ hybridization with mKast (Kaneko et al., 2013).

#### sKCs

Three genes, ecdysone receptor (EcR), hormone receptor-like 38 (HR38), and E74, are expressed preferentially in the sKCs, and all of them encode transcription factors involved in the ecdysteroidsignaling pathway (**Figure 1D**; Paul et al., 2005; Yamazaki et al., 2006; Takeuchi et al., 2007). Expression of HR38 is higher in the brains of foragers than in the brains of nurse bees, suggesting its possible association with the division of labor of workers (Yamazaki et al., 2006). The EcR/ultraspiracle (Usp) heterodimer binds to ecdysteroids to orchestrate transcriptional regulation during metamorphosis (Davis et al., 2005). In contrast, HR38 competes with EcR for Usp, and the HR38/Usp heterodimer activates the transcription of target genes distinct from those of the EcR/Usp heterodimer (Zhu et al., 2000; Baker et al., 2003). Thus, Yamazaki et al. (2006) previously proposed that the enhanced expression of HR38 in the forager brain might contribute to switching the mode of ecdysteroid-signaling in the MBs from the EcR- to the HR38-mediated pathway in association with the division of labor of workers (Yamazaki et al., 2006).

Recent studies, however, reported that, in the silk moth and fruit fly, HR38 is an immediate early gene, whose neuronal expression is activated by neuronal excitation (Fujita et al., 2013), and that HR38 expression in the honeybee brain is induced by foraging behavior (Ugajin et al., 2018). These results suggest an alternative possibility that HR38 expression in the sKCs of the honeybee brain is a consequence of the foraging behavior, and does not necessarily represent a gene expression profile specific to the forager brain. These possibilities need to be investigated further.

On the contrary, Gehring et al. (2016) reported that phosphorylated (activated) cAMP-response element binding protein (pCREB) is enriched in the sKCs in honeybee MBs (**Figure 1D**; Gehring et al., 2016), suggesting that the sKCs are related to CREB-based memory function (McGuire et al., 2005; Alberini, 2009; Hirano et al., 2016).

#### mKCs

So far, only one gene, termed mKast, has been found to be expressed preferentially in the mKCs of the honeybee MBs (**Figures 1C,D**) (Kaneko et al., 2013). Although mKast belongs to the α-arrestin family, which is involved in downregulation of membrane receptors (Kaneko et al., 2013), the role of mKast in the honeybee is currently obscure. mKast expression in the brain begins at the late pupal stages and is detectable almost exclusively in the adult brain, suggesting its role in regulating adult honeybee behaviors and/or physiology (Yamane et al., 2017).

Since detection of neural activity using immediate early genes revealed that MB KCs (Singh et al., 2018; Ugajin et al., 2018), especially sKCs and some mKCs (Kaneko et al., 2013), are active in the brains of foragers, it is plausible that these KC subtypes are related to sensory information processing during the foraging flights.

#### Broader Gene Expression Profiles

Three genes, PLCe (Suenami et al., 2016), protein kinase C (PKC) (Kamikouchi et al., 2000), and E75 (Paul et al., 2006), are preferentially expressed in all KC subtypes (=the whole MBs) in the honeybee brain (**Figure 1D**). Considering that E75 is expressed preferentially in all KC subtypes (=the whole MBs) (Paul et al., 2006), whereas EcR, HR38, and E74 are preferentially expressed in the sKCs (Paul et al., 2005; Yamazaki et al., 2006; Takeuchi et al., 2007), it might be that different ecdysteroidsignaling pathways function in distinct KC subtypes.

With regards to PLC, there are four homologs, including PLCe, in the honeybee. The PLCe is expressed almost selectively in the whole MBs, and expression of the other three homologs is significantly higher in the MBs than in other brain regions (Suenami et al., 2017). Suenami et al. (2017) revealed that pharmacological inhibition of PLC significantly attenuated the memory acquisition, but did not affect memory retention, suggesting that PLCs are involved in early memory formation in the honeybee (Suenami et al., 2017). Thus, although both CaMKII and PLC are involved in Ca2+-signaling (Smrcka et al., 2012; Dusaban and Brown, 2015), they play roles at different stages of learning and memory. It will be interesting to test whether their roles at different stages of learning and memory can be attributed to their distinct KC subtype-preferential expression.

Two genes, tachykinin-related peptide (Trp) and juvenile hormone diol kinase (JHDK), are preferentially expressed in both the "redefined" lKCs and "redefined" sKCs, but not in the mKCs (**Figure 1D**; Takeuchi et al., 2004; Uno et al., 2007; Kaneko et al., 2013). The Trps are multifunctional brain/gut peptides that have important roles in neurotransmission and/or neuromodulation (Van Loy et al., 2010). In Drosophila, tachykinin-expressing neurons control male-specific aggressive behaviors (Asahina et al., 2010). Therefore, it might be that Trp is also involved in the control of aggressive behaviors even in the honeybee. The function of JHDK in insects is not well understood (Uno et al., 2007).

Interestingly, McQuillan et al. (2012) reported that the expression of genes for amine receptors, which are involved in learning and memory, differs across KC subpopulations (McQuillan et al., 2012), which is consistent with the recent notion that different regions of the MBs contribute to learning and memory in Drosophila (Zars et al., 2001; McGuire et al., 2003; Trannoy et al., 2011).

# KC Population Expressing FoxP

Recently, Schatton and Scharff (2017); Schatton et al. (2018) identified a novel KC population expressing transcription factor FoxP in the MBs of the honeybee brain (**Figure 1D**) (Schatton and Scharff, 2017; Schatton et al., 2018). Although Kiya et al. (2008), who first reported the FoxP expression in the honeybee brain, detected no significant FoxP expression in the honeybee MBs (Kiya et al., 2008), Schatton et al. notified that, in Drosophila, a MB-core subpopulation expresses FoxP, which is related to decision-making (DasGupta et al., 2014). They also reported FoxP expression in the honeybee MBs (Schatton and Scharff, 2017). These findings suggest that neural populations with FoxP expression that are related to reinforcement-based learning abilities are conserved among animal species (Schatton and Scharff, 2017; Schatton et al., 2018).

There seems to be a problem, however: although Schatton et al. indicated that the KC population expressing FoxP does not overlap with mKCs, and speculated that FoxP specifies different subsets of mKC (Schatton and Scharff, 2017), Kaneko et al. (2013) and Suenami et al. (2016) reported that lKCs do not overlap with mKCs, and observed no gaps between the areas where lKC and mKC somata exist (Kaneko et al., 2013; Suenami et al., 2016). Based on the latter findings, the KC population expressing FoxP is assumed to be the lKCs. This point needs to be clarified in future studies.

# ANALYSIS OF KC SUBTYPE DIFFERENTIATION DURING METAMORPHOSIS

Genes expressed in a KC subtype-preferential manner can be used as markers to trace the differentiation of KC subtypes or their evolution in hymenopteran insects.

In honeybees, larval MBs comprise only class II KCs. Class I "classical" lKCs and sKCs are newly produced from proliferating neuroblasts whose somata are located in the inner core inside of the MB calyces during the pupal stages (Farris et al., 1999) and cease their proliferation at the P2 and P5 stages, respectively. Suenami et al. (2016) recently used three genes, Syt14, dlg5, and PLCe, as markers to trace the differentiation of the "redefined"

lKC (Syt14, and dlg5) and all KC subtypes (PLCe) (Suenami et al., 2016). The PLCe is already expressed in larval MBs and continues to be expressed in the whole MBs during the pupal stages, suggesting that Ca2+-signaling is enhanced in the whole MBs during the entire honeybee lifespan. The expression of Syt14 and dlg5 becomes detectable at the middle pupal stages (around P3), and is restricted to the lKCs at the adult stage, suggesting that expression of Syt14 and dlg5 is characteristic of differentiated lKCs (Suenami et al., 2016). The FoxP expression is also not detected in larval MBs, but becomes detectable in the MBs at the middle-to-late pupal stages (P4-5) (Schatton et al., 2018), suggesting that FoxP expression is also characteristic of differentiated KCs. In contrast, KCs expressing mKast become detectable at the late pupal stages (P7 and P8) (Kaneko et al., 2013), suggesting that mKCs develop after the lKCs begin to differentiate or mKast is expressed at the late stage of mKC differentiation.

# POSSIBLE KC SUBTYPE EVOLUTION IN HYMENOPTERAN INSECTS

Farris and Schulmeister (2011) indicated that both aculeate insects and parasitic wasps, which are hymenopteran insects that appeared later in the course of evolution, have more morphologically elaborate MB calyces than sawflies, which are primitive hymenopteran insects, and proposed that the elaborate MB calyces are associated with the higher learning ability of parasitic wasps (Farris and Schulmeister, 2011). This leads to the question of when during the evolution of hymenopteran insects were KC subtypes acquired? To address this question, Oya et al. (2017) performed in situ hybridization of Trp homologs to compare KC subtypes among the brains of four hymenopteran insect species: (1) a phytophagous and solitary sawfly (Symphyta; Arge similis), (2) a solitary parasitic wasp (Apocrita; Ascogaster reticulata), (3) an eusocial hornet (Aculeata; Vespa mandarinia), and (4) a nidificating and solitary scoliid wasp (Aculeata; Campsomeris prismatica) (Oya et al., 2017). As Trp is expressed in both "redefined" lKCs and "redefined" sKCs, but not in mKCs; the presence of all three KC subtypes can be visualized in a certain hymenopteran insect brain by performing in situ hybridization of a single Trp homolog (Takeuchi et al., 2004).

The brains of V. mandarinia and C. prismatica have three class I KC subtypes (lKCs, mKCs, and sKCs), as observed in the honeybee. In contrast, the brain of A. reticulata has only two KC subtypes; "large" KCs with significant Trp-expression and "small" KCs with no significant Trp-expression, and the brain of the sawfly A. similis has no discriminable KC subtypes (Farris and Schulmeister, 2011) (**Figure 2A**). Discrimination of class I and II KCs is difficult in A. reticulata and A. similis, because the MB calyces are shallow and Class I and II KCs seem to be merged in these species.

It is plausible that the advanced learning abilities of parasitic wasps to search for their host insects require MBs with elaborate calyces and both ancestral (original) and second KC subtypes, whereas the highly advanced learning abilities of aculeate insects to return to their nests require MBs with all of the class I KC subtypes, in addition to the elaborate MB calyces (Whitfield, 2003; Huber, 2009; Johnson et al., 2013). To test this notion, the correspondence between one and two KC subtypes detected in sawfly and parasitic wasps, and three KC subtypes detected in aculeate insects will need to be examined by in situ hybridization for homologs of genes expressed in a KC subtype-preferential manner in the honeybee (e.g., Syt14, dlg5, or Mblk-1 for "redefined" lKCs; mKast for mKCs; and Trp or JHDK for "redefined" lKCs/sKCs, respectively. See also **Figure 1D**) (Kubo, 2012; Kaneko et al., 2013, 2016; Suenami et al., 2016). The KC subtype/population that expresses FoxP in these hymenopteran insect species is also an intriguing topic for future investigation (Schatton and Scharff, 2017). Such experiments are expected to unveil KC subtype/population of ancestor origin in the hymenopteran insects and those unique to aculeate insects.

#### APPLICATION OF GENOME EDITING FOR ANALYSIS OF THE ROLE OF KC SUBTYPES IN THE HONEYBEE

While RNAi is effective for analyzing gene function, its efficiency sometimes varies depending on the animal species and target genes and/or organs (Matsumoto et al., 2014). In addition, it is difficult to suppress gene function for a long time (Matsumoto et al., 2014). An alternative method for the analysis of gene function is genome editing. Genome editing has been applied to some hymenopteran insects, including the sawfly Athalia rosae (Hatakeyama et al., 2016), parasitic wasp Nasonia vitripennis (Li et al., 2017), and two social ants, Ooceraea biroi and Harpegnathos

#### REFERENCES


saltator (Trible et al., 2017; Yan et al., 2017). A transgenic technique using piggyBac has been applied to honeybees (Schulte et al., 2014). Recently, Kohno et al. (2016) established a basic genome-editing technique in the honeybee to analyze in vivo gene function (Kohno et al., 2016).

To analyze the roles of genes in regulating the behaviors and/or brain functions exhibited by honeybee workers, it is necessary to produce hetero- or homozygous mutant workers (F3) through several steps (**Figure 2B**; Kohno et al., 2016). For this, it is important that adult mutant honeybees [mutant drones (F1) and homozygous mutant workers (F3)] should be alive; in other words, the target gene(s) must be dispensable for normal development and sexual maturation in honeybees. Kohno et al. selected major royal jelly protein 1 (mrjp1) as a target gene to establish basic honeybee genome-editing techniques. The MRJP1 is the most abundant protein component of the royal jelly, which is produced by the hypopharyngeal glands of young nurse bees and secreted as food for the larvae, drones, and queens (Kubo et al., 1996; Ohashi et al., 1997; Schmitzová et al., 1998). As expected, the results indicated that mrjp1 is dispensable for normal drone development (Kohno et al., 2016).

Genes expressed in a KC subtype-preferential manner can also be good candidate target genes for genome editing, because some of them are assumed to relate to some brain functions and some of them are dispensable for normal honeybee development and sexual maturation. Investigation of the functions of genes involved in development and sexual maturation will require other methods as well, such as the expression of knocked-in genes in a stage- and/or tissue-specific manner by genome-editing.

#### AUTHOR CONTRIBUTIONS

TK drafted the manuscript and figures. SS, SO, HK, and TK wrote and reviewed the manuscript, and completed figures.

# FUNDING

This work was supported by a Grant-in-Aid for JSPS Fellows (17J03716).


Zhu, J., Miura, K., Chen, L., and Raikhel, A. S. (2000). AHR38, a homolog of NGFI-B, inhibits formation of the functional ecdysteroid receptor in the mosquito Aedes aegypti. EMBO J. 19, 253–262. doi: 10.1093/emboj/19.2.253

**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2018 Suenami, Oya, Kohno and Kubo. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# The Central Complex as a Potential Substrate for Vector Based Navigation

Florent Le Moël <sup>1</sup> , Thomas Stone<sup>2</sup> , Mathieu Lihoreau<sup>1</sup> , Antoine Wystrach<sup>1</sup> \* † and Barbara Webb<sup>2</sup> \* †

<sup>1</sup> Research Centre on Animal Cognition, Centre for Integrative Biology, CNRS, University of Toulouse, Toulouse, France, <sup>2</sup> School of Informatics, University of Edinburgh, Edinburgh, United Kingdom

#### Edited by:

Jeffrey A. Riffell, University of Washington, United States

#### Reviewed by:

Thierry Hoinville, Bielefeld University, Germany Nicholas Kathman, Case Western Reserve University, United States

#### \*Correspondence:

Antoine Wystrach antoine.wystrach@univ-tlse3.fr Barbara Webb b.Webb@ed.ac.uk

†Co-last authors

#### Specialty section:

This article was submitted to Comparative Psychology, a section of the journal Frontiers in Psychology

Received: 29 March 2018 Accepted: 12 March 2019 Published: 05 April 2019

#### Citation:

Le Moël F, Stone T, Lihoreau M, Wystrach A and Webb B (2019) The Central Complex as a Potential Substrate for Vector Based Navigation. Front. Psychol. 10:690. doi: 10.3389/fpsyg.2019.00690 Insects use path integration (PI) to maintain a home vector, but can also store and recall vector-memories that take them from home to a food location, and even allow them to take novel shortcuts between food locations. The neural circuit of the Central Complex (a brain area that receives compass and optic flow information) forms a plausible substrate for these behaviors. A recent model, grounded in neurophysiological and neuroanatomical data, can account for PI during outbound exploratory routes and the control of steering to return home. Here, we show that minor, hypothetical but neurally plausible, extensions of this model can additionally explain how insects could store and recall PI vectors to follow food-ward paths, take shortcuts, search at the feeder and re-calibrate their vector-memories with experience. In addition, a simple assumption about how one of multiple vector-memories might be chosen at any point in time can produce the development and maintenance of efficient routes between multiple locations, as observed in bees. The central complex circuitry is therefore well-suited to allow for a rich vector-based navigational repertoire.

Keywords: vector, path integration, memory, insect, navigation, neural modeling, traplining, central complex

# 1. INTRODUCTION

It is well established that central place foraging insects, such as bees and ants, keep track of their displacement when they venture outside their nest by a process called path integration (PI) (Collett and Collett, 2000a,b). By combining compass and speed information, they continuously update a home vector that allows for a direct return to their nest after arbitrary outward routes (Müller and Wehner, 1988; Collett and Collett, 2000b). However, insects do not use their PI system only for homing. For instance, they can also store PI vector-memories and use them to return to a known food location (Wehner et al., 1983; Collett et al., 1999; Wolf and Wehner, 2000), and take shortcuts between multiple food locations (Menzel et al., 2005).

A recently published neural model (Stone et al., 2017) closely follows the connectivity of the insect Central Complex neuropil (CX) and uses properties of identified neurons in this circuit that respond to polarized light compass information and optic flow information to integrate an outbound path. In this model, the home vector, at any point in time, is assumed to exist as a distributed sinusoidal activity pattern across two sets of 8 columns, where the phase indicates direction, and amplitude indicates distance. The model also provides a mechanism for using such a PI memory to drive the animal directly back home. Offset connections between columns produce a comparison of the current heading to the home vector direction, and indicate whether steering left or right would improve the alignment. As the circuit continues to integrate movement, the home vector amplitude will decrease as it approaches the home position. When it becomes zero, an emergent search behavior will result, unless there is a mechanism to recognize home. The model accounts for changing travel speed and is also robust to decoupling between the agent body axis and direction of movement (Stone et al., 2017), something that bees (Riley et al., 1999), wasps (Stürzl et al., 2016) and ants (Pfeffer and Wittlinger, 2016; Collett et al., 2017; Schwarz et al., 2017) can do.

The steering mechanism in this model is generalizable beyond the use of a home vector. Different sources of information about the "desired" heading or destination could be switched in, or additively combined onto the steering neurons, and the system will automatically steer to reduce the difference between the current and desired directions. While it is interesting to speculate how this might include information from sources other than PI (e.g., learnt terrestrial cues), here we focus on cases where the alternative activation is derived from a "vector-memory." That is, we assume that, as in other models (Cruse and Wehner, 2011; Hoinville et al., 2012), the animal can store the current state of its home vector (the neural activation pattern) when it encounters salient places in its environment, and can later recover this vector-memory to guide future behavior (**Figure 1A**). We suggest some simple (hypothetical) neural circuitry that would add this capability to the CX model (**Figure 1B**) (we assess its biological plausibility in the discussion) and show it can support several interesting phenomena observed in insect navigation.

Memory-directed movement: Insects that have found a food source on a previous excursion can return to it on a direct route. It is assumed this involves storage of a memory of the PI state when the food was reached (Wehner et al., 1983; Collett et al., 1999; Wolf and Wehner, 2000). We hypothesize that such a memory could be integrated as a simple inhibitory influence in the CX steering circuit to produce food-ward steering and search around the food location (**Figures 2**, **3**).

Vector-memory re-calibration: Insects experiencing a PI inconsistency when returning from food to the nest due to a forced displacement, appear to make a partial adjustment of their memory of the food location (Collett et al., 1999; Wehner et al., 2002; Bolek et al., 2012) (although the extent of this "recalibration" seems to vary with experimental conditions). We suggest how this updating of a food-ward vector-memory could occur (**Figure 4**).

Shortcutting: Bees have been observed to make novel shortcuts between remembered food locations (Menzel et al., 2005). It has previously been demonstrated that this can be obtained by vector addition, i.e., combining the current state of the home vector (from an arbitrary location such as a first food source) with a vector-memory from home to another food source (Cruse and Wehner, 2011). This produces a vector directly from the current location to the food. We show that such shortcutting would be a straightforward consequence of switching between memories in the CX circuit; importantly, this demonstrates how vector addition could be implemented in the insect brain (**Figure 5**).

Multi-location routes: Bees often feed on multiple locations (e.g., feeders or flowers patches) before returning home, and have been shown to take efficient multi-location routes, or "traplines," that minimize the overall journey distance (Ohashi et al., 2006; Lihoreau et al., 2012b; Buatois and Lihoreau, 2016). We investigate a simple rule by which the neural circuit output can be used to choose the next location to visit, and test whether this produces multi-location routes similar to bees (**Figure 6**).

Route ontogeny: Finally, we explore how such multi-location routes might develop over repeated foraging excursions through a combination of random exploration and vector-memory recall (**Figure 7**).

# 2. METHODS

# 2.1. Environment and Agent

We simulate (using Python 2.7) an agent moving in a 2D environment. Movement in these simulations is discretised in time and space. Units are therefore arbitrary, and different walking "speeds" may be achieved by changing the length of the spatial step that the agent moves at a time. In the following paper, we describe the agent's movement as time steps (t), where the "speed" is generally kept constant during tests, but variable during random walks (see **Supplementary Material** section "Random Walks"). The environment typically contains a nest, one or multiple feeders, as well as optional obstacles. The nest and feeders are circular with a small defined radius (relative to the typical environment size) within which the agent is assumed to have "landed" successfully at the target, and a larger radius, or "catchment area" which is assumed to provide an olfactory signal (or other attractive signal) that could steer the agent to the target. Obstacles can have circular, rectangular or wall-like shapes and prevent the agent from passing through the area they occupy (e.g., walls enclosing the agent in an arena) by emitting a very short range repulsion signal that can steer the agent away.

The agent's size is one spatial unit. It is assumed to have sensory information about its heading direction in an absolute external reference frame, as could be supplied in real insects for example by a celestial compass (over a short time duration, or with internal clock correction, Labhart and Meyer, 2002). It is also assumed to have information about its instantaneous speed of movement in its heading direction as could be supplied by optic flow, step counting, or efference copy. These provide inputs to the CX model for path integration and control of steering. Lastly, the agent is equipped with two "detectors," oriented at 90 degrees, that provide no input whatsoever to the neural model we describe, but only act as modulators of the agent's turning intensity in response to "attraction" or "repulsion" signals emitted by objects in the environment such as the nest, feeders, or obstacles.

The agent's starting position for each simulation is (unless specified otherwise) set at the nest. Its position is updated iteratively depending on its speed v and heading θ:

$$\begin{cases} \varkappa\_t = \varkappa\_{t-1} + \nu\_{t-1} \cos(\theta\_{t-1}) \\ \varkappa\_t = \varkappa\_{t-1} + \nu\_{t-1} \sin(\theta\_{t-1}) \end{cases} \tag{1}$$

The speed and heading can be controlled by a random walk process (see "Random Walk" section in **Supplementary Material**) or have a fixed speed (v<sup>t</sup> = 0.15) and a heading given by the outputs of the CX steering neurons (see section 2.2), depending on a flag that sets the current motivational state (see below). Or, when an obstacle or a goal is detected, the heading is given as follows:

$$\begin{array}{l} \mathcal{M}\_{left} \propto \text{(}\mathcal{R}\_{left}A\_{obj}\text{)}\\ \mathcal{M}\_{right} \propto \text{(}\mathcal{R}\_{right}A\_{obj}\text{)} \end{array} \tag{2}$$

$$\theta = (M\_{right} - M\_{left}) + noise \tag{3}$$

with Mleft and Mright the modulation for left and right sides, respectively, which are proportional to the left and right readings Rleft and Rright of the two detectors, multiplied by the detected object's attractiveness Aobj. The added noise is drawn from a VonMises distribution centered on 0:

$$noise \sim vonMiss(0, \kappa)\tag{4}$$

where κ = 100.0 is the concentration of the VonMises distribution. Note that this is considered to be a basic reflex behavior of the agent, which by-passes the CX circuit. Finally, in such case of a environment-driven steering modulation, the agent's speed is also modulated by an increased drag value (multiplied by a factor of 1.5), providing better turns.

#### 2.2. Central Complex Model

For convenience, we provide here an overview of the mathematical description of the CX model, but we deliberately omit the detailed biological justification, which is covered at length in Stone et al. (2017). Layers 1–4 are identical to the previous model. A "vector-memory" neuron has been added, which can store the output state of layer 4, and in turn, modulate this output before it reaches layer 5 (steering).

In overview, the circuit consists of a set of direction cells (layer 3) that divide the azimuthal space and are activated by the current heading of the agent (layers 1 & 2). Mutual inhibition in layer 3 forms a ring attractor circuit creating a stable distributed pattern in the form of a sinusoid. A set of integrator cells (layer 4) receive speed input but are inhibited by their corresponding direction cells and thus accumulate distance traveled opposite to the heading direction, creating a distributed representation of the home vector. The vector-memory allows the current state of the home vector to be stored when the agent is at salient locations (feeders). The state is stored in the synaptic weights of one neuron for each memory location. Homing is controlled by steering cells (layer 5) that compare the integrator cell activation to the current direction cell activation to determine if the animal should turn left or right. Vector-memory can be used to selectively influence this comparison process.

This circuit uses firing rate model neurons, in which the output firing rate r is a sigmoid function of the input I:

$$r = \frac{1}{\left(1 + e^{-(aI - b)}\right)}\tag{5}$$

where parameters a and b control the slope and offset of the sigmoid. On this value is added a Gaussian noise N(0, σ 2 r ), with σ = 0.1. This output firing rate is, across all layers, subject to a clipping between 0 and 1 to prevent the applied noise to depart from the range [0, 1]. The input I is given by the weighted sum of activity of neurons that synapse onto neuron j:

$$I\_{\dot{j}} = \sum\_{i} W\_{\dot{j}\dot{j}} r\_i \tag{6}$$

The value of the parameters for slope, offset and connection weights for each layer are provided in **Supplementary Material**.

#### 2.2.1. Layer 1 - Speed Input

To implement input to our speed-sensing (TN2) neurons, we simulate forward-to-backward optic flow sensing, taking into account the diagonally offset preferred angles of identified TNcells in the CX noduli in each hemisphere (Stone et al., 2017):

$$\begin{array}{l} I\_{T\text{N}\_L} = [\cos(\theta + \phi), \sin(\theta + \phi)] \cdot \nu \\ I\_{T\text{N}\_R} = [\cos(\theta - \phi), \sin(\theta - \phi)] \cdot \nu \end{array} \tag{7}$$

where **v** is the velocity vector of the agent, · the dot product, θ ∈ [0, 2π) is the current heading of the agent and φ is the preferred angle of a TN-neuron, i.e., the point of expansion of optic flow that evokes the biggest response. For our model, a default preferred angle of φ = (π/4) was used. TN2 neurons have their value clipped between 0 and 1 so that they respond in a positive linearly proportional manner to ITN, but have no response to negative flow (backward motion):

$$r\_{TN2} = \min(1, \max(0, I\_{TN})) \tag{8}$$

In practice for this paper we assume that the agent is moving in the direction it is facing, i.e., **v** = [cos(θ),sin(θ)]v, which will produce an equal response in each TN2 neuron, i.e., ITN<sup>L</sup> = ITN<sup>R</sup> = cos(φ)v regardless of the heading θ.

#### 2.2.2. Layer 1 - Directional Input

The first layer of Directional input consists of 16 input neurons, each of which has a preferred direction α ∈ {0, π/4, π/2, 3π/4, π, 5π/4, 3π/2, 7π/4} with each of the 8 cardinal directions represented twice over. We identify these with polarization sensitive TL neurons in the insect central complex (Stone et al., 2017). On each time step they receive input corresponding to the cosine of the difference between their preferred heading and the agent's current heading θ ∈ [0, 2π):

$$I\_{TL} = \cos(\alpha - \theta) \tag{9}$$

#### 2.2.3. Layer 2

The second layer consists of 16 neurons that receive inhibitory input proportional to the output of the first directional input layer. This simple inversion of the response across the array is not actually crucial but is included to model the properties observed in CL1 neurons connecting the polarization input to the protocerebral bridge (Stone et al., 2017).

$$I\_{\rm CL1} = -r\_{\rm TL} \tag{10}$$

#### 2.2.4. Layer 3 - Compass

The third layer consists of 8 neurons that get input from each pair of CL neurons that have the same directional preference. These neurons are identified with the TB1 neurons in the protocerebral bridge of the CX, which also make mutually inhibitory connections with each other in a specific pattern that resembles a ring-attractor circuit (Stone et al., 2017). Thus, their input is given by:

$$I\_{TB1} = \mathcal{W}\_{CL1,TB1} r\_{CL1} + \mathcal{W}\_{TB1,TB1} r\_{TB1} \tag{11}$$

where WCL1,TB<sup>1</sup> is a [0, 1] matrix mapping pairs of CL neurons to single TB1 neurons, and WTB1,TB<sup>1</sup> is a matrix of inhibitory weights between TB1 neurons where:

$$W\_{TB1\_{i,TB1\_j}} = \frac{d(\cos(\alpha\_i - \alpha\_j) - 1)}{2} \tag{12}$$

where α<sup>i</sup> and α<sup>j</sup> are the preferred directions of their respective TB1 inputs, and d = 0.33 is a scaling factor for the relative effect of this inhibition compared to the direct CL1 excitation.

#### 2.2.5. Layer 4 - Speed Accumulation

The fourth layer consists of 16 neurons, which we associate with the CPU4 cells that occur in each column of the CX central body upper. These receive input from both the protocerebral bridge (TB1) and the noduli (TN2). The input for these neurons is an accumulation of heading of the agent, obtained by inhibitory compass modulation of the speed signal from the speed-sensitive neurons:

$$I\_{CPU4\_t} = I\_{CPU4\_{t-1}} + acc \times (r\_{TN2\_t} - r\_{TB1\_t} - decay) \tag{13}$$

where rTN<sup>2</sup> is the speed-sensitive response, rTB<sup>1</sup> the compasssensitive response; and acc = 0.0025 and decay = 0.1 determine the relative rates of memory accumulation and memory loss. The charge of all integrator cells starts at ICPU4t<sup>0</sup> = 0.5 and, as it accumulates, is clipped on each time step to fall between 0 and 1. Note that accumulation occurs on the input, i.e., it is not affected by the non-linearity of the neuron's output function. Also note that the decay shifts the whole activity pattern toward 0, rather than moving the relative amplitude in each accumulator toward the others. As such, this does not act as a leaky integration of the path (as proposed in e.g., Sommer and Wehner, 2004 and as modeled in e.g., Vickerstaff and Di Paolo, 2005), as the relative amplitude will still encode the veridical home vector, unless the leak (or the accumulation) are enough to cause the values to be clipped at 0 (or 1). The 8 TB1 neurons each provide input to two CPU4 neurons which will thus have identical activity (other than added random noise, see below) as we assume the agent moves in its heading direction thus generating symmetric optic flow. As these neurons integrate the velocity (i.e., speed and direction) of the agent, the activity across this layer at any point in time provides a population encoding of the home vector.

#### 2.2.6. Vector-Memory

This is the only new component in circuit compared to Stone et al. (2017). It is a hypothetical addition and as yet we do not suggest any specific identified neural analog. We store the vector-memory in the synaptic weights of a hypothetical memory neuron that inhibits the output of the CPU4 integrator cells: i.e., the memory neuron has 16 inhibitory output synapses, one per CPU4 output fiber (see **Figures 1B**, **2A**).

The weight of these synapses are set according to the corresponding activity of the CPU4 output fiber at the moment of learning, as could be signaled by a reinforcer neuron. More precisely, we store the ICPU<sup>4</sup> values after passing through a sigmoid function of the same slope and bias parameters as the CPU4 response (see **Supplementary Material**, "Neurons parameters"), but without any added noise. This is to avoid encoding the instantaneous noise level (i.e., the one of the last time step only), and can be interpreted as the learning taking place over a short time interval to more precisely estimate the current CPU4 activity. The noise is then added dynamically (at each time step) during recall, like in the rest of the system. The obtained values are negated in sign (since the synapses are inhibitory). In other words, the agent's current home vector gets stored in the 16 synaptic weights of the memory neuron when the reinforcer neuron is triggered (**Figure 1D**). The learning of the vector-memories is set at particular time or locations: in this paper, these are associated with the discovery of food. As described below, this will allow the agent to return to the position at which the vector was stored. For some experiments we allow the agent to store more than one such vector-memory, into separate memory neurons, corresponding to different food locations.

Thus, the vector-memory synapses can be represented as a 16-values vector WVM:

$$W\_{VM} = -\begin{cases} r\_{CPU4\_{noics}}, & \text{if signal to store} \\ baseline, & \text{otherwise} \end{cases} \tag{14}$$

with baseline being a vector of 16 zero-state values (= 0.5, since firing rate is encoded between 0 and 1).

#### 2.2.7. Vector-Memory Recalibration

We also introduce a potential re-calibration of the vectormemories, based on the state of Layer 4 when the agent reaches the nest. In the absence of error (either noise or induced through an experimental manipulation) this state should be zero, so any remaining activation in the Layer 4 thus encodes a possible "error vector" accumulated across the whole path (inbound and/or outbound).

This "error vector" can be used to modulate the vectormemory synapses. For this, another hypothetical process very similar to the learning described above, is used: a "recalibrator" neuron, triggered when the agent arrives at the nest, modulates the vector-memory synapses that were last active, similarly to the reinforcer neuron used for learning, only differing in the sign of the modulation. That way, the potential "error vector" remaining in the CPU4 population causes the re-calibration of the last active vector-memory (**Figure 4A**).

Thus, the vector-memory WVM update:

$$W\_{VM\_{\text{rec}}} = W\_{VM} + r\_{\text{rec}}(b - r\_{\text{CPU}4\_N}) \tag{15}$$

where baseline b = 0.5, rCPU4<sup>N</sup> is the output of the integrator when the Nest is reached,rrec is the activation of the "recalibrator" neuron, or in other words the efficiency of this re-calibration. For instance, with an efficiency rrec = 1, the updated vector-memory will be fully corrected for the error. For rrec = 0.5 the result will be an average between the previously stored vector-memory and a fully error-corrected one.

#### 2.2.8. Layer 5 - Steering Output

This layer contains 16 neurons which receive input from the compass (layer 3), and the home vector (layer 4) modulated by the vector-memory neuron. These inputs can be switched on or off depending on the agent's state, e.g., whether it is attempting to return home or to return to the location where a vector was stored. The input from the compass layer 3 is inhibitory, following the same pattern as the layer 3 to layer 4 connections. The connections from layer 4 to layer 5 are offset, by one column to the left for one set of 8 neurons CPU1L, and by one column to the right for the other set of 8 neurons CPU1R. The vectormemory synapses modulate the output from layer 4 to layer 5.

We identify the steering neurons with the CPU1 neurons in the central body upper of the CX, which anatomically reveal the offset pattern used in the model. Inside layer 5 are also pontine neurons that receive the same pattern of input from layer 4, and provide inhibitory output that balances and filters the activity across both hemispheres (see Stone et al., 2017 for more detail). For convenience we neglect the pontine neurons in the equation below because they do not affect the circuit when using symmetric speed input:

Note first that in the "exploring" state, the left and right activity will be identical and hence will not affect the steering. In the "homing" state, the circuit effectively performs a comparison of the population vectors representing current heading (compass) (TB1) and the integrator CPU4, but the connectivity pattern between the integrator and the steering cells means that the desired heading signal is offset in both directions by one column. Hence the left and right activity of the steering cells will represent whether the left or right offset provides a better alignment, and the difference between them can be used to steer, as described in Equation (17). As the integrator keeps running, the steering signal will disappear (or be dominated by noise) when the agent nears home, producing a search pattern.

In the "using vector-memory" state, the output of the integrator is balanced by inhibition from a vector-memory stored at a feeder location (see above). If starting from the nest, with the integrator containing a zero home vector, this negative influence means the agent acts as though its own location (for the purpose of steering) is exactly opposite to where the feeder is located, and the steering circuit will drive it "home" from its actual location (the nest) toward the food. Since the path integration continues to run in parallel, accurately reflecting the agent's actual displacement, when the food location is reached the input from the integrator to the steering layer will cancel out the negative influence from the vector-memory and the agent will start its search pattern, just as it would at the end of a regular "homing" state.

# 2.3. Experimental Paradigms

#### 2.3.1. Memory-Directed Movement

To observe the efficiency of the memory-directed movements, the task is realized in two parts: First, the agent performed random walks of different lengths, originating from the nest (x = 0, y = 0), and stored for each of these the final integrator state as a new vector-memory. Then, after being reset to the nest (coordinates reset to x = 0, y = 0; integrator reset to baseline = 0.5), a vector-memory was recalled and allowed to drive the behavior. We used a feeder catchment area of 20-steps


where WCPU4,CPU<sup>1</sup> is the connectivity matrix from CPU4 to CPU1 cells, WVM is synapses weight vector of the vectormemory and rVM is the activation of a specific vector-memory neuron (basically rVM = 1 when using that vector-memory, rVM = 0 otherwise).

The output of CPU1 cells project to the left and right lateral accessory lobes, which are pre-motor centers. We thus use the difference in CPU1<sup>L</sup> and CPU1<sup>R</sup> sets to provide a steering signal for the agent:

$$\theta\_{\ell} = \theta\_{\ell-1} + 0.5(\sum\_{i=1}^{8} r\_{\text{CPU}1\_{Li}} - \sum\_{i=1}^{8} r\_{\text{CPU}1\_{Ri}}) \tag{17}$$

radius: as soon as the agent entered the feeder catchment area, its proximity sensors guided it to the feeder location. We typically ran N = 1, 000 trials at 20 random-walk lengths, equally spaced between 100 and 10,000 steps.

A basic measure used was the proportion of successful trials. We considered a food-ward route successful if the agent reached the feeder coordinates within a given time limit of 5,000 steps. It is expected that the agent reaches the target in a straighter path and then performs random search around the expected location. We also evaluated the systematic search patterns produced, either by an agent returning home after a random walk, or an agent using a vector-memory from the nest location to return to the food (see "Systematic search" section in **Supplementary Material**). In this

FIGURE 1 | Basis of the concept of inhibition by Vector memory. (A) Example of the vector-memory and shortcut rationale: 0. The agent found a feeder (F2) on a previous trip and stored the corresponding home vector (solid purple) as a vector-memory. (1a) The agent leaves the Nest, performs a random walk (solid gray), and finds the feeder (F1). (1b) It stores the home vector (solid green) as a vector-memory. (1c) It uses the home vector to return to the Nest. (2a) The agent recalls the F1 vector-memory, "imagining" it is on the far side while actually at home (dashed green). (2b) It tries to "home" (dashed orange) which means it actually moves back to F1 (solid orange). (3a) At F1, no food is found: it lifts the recall of the F1 vector-memory and recalls the F2 vector-memory instead (dashed purple). (3b) It thus tries to "home" in a new direction (dashed red) which results in an actual movement from F1 to F2 (solid red). Lifting the F2 vector-memory recall allows it to home correctly (solid purple). (B) Principal connections of all cell types included in the Central Complex model: Shown are all connections of one direction cell (TB1), irrespective of columnar identity of individual cells (only two out of six connections to other TB1 cells are shown). The vector-memory neuron shows inhibitory synapses to the output fibers of the integrator (CPU4) cells, each of these synapses' weight being set according the corresponding CPU4 cell activity at the time of learning. (C) Example snapshots of the population activity of the 16 integrator (CPU4) neurons, at two different positions, with or without vector-memory recall: Solid lines thus correspond (Continued)

FIGURE 1 | to the output of the integrator, dashed lines to the output of the integrator under the effect of a vector-memory neuron. At the Nest (solid blue), the integrator is in the zero-state (flat line). At the feeder (solid orange), the integrator encodes the position in polar coordinates across the population: sinusoid amplitude is the distance, phase is the angle. Under the inhibition by the vector-memory neuron, when the agent is at the Nest (dashed blue) the apparent coordinates encode for the Nest-to-Feeder vector. At the feeder, still under the effect of the vector-memory neuron (dashed orange), the integrator output and the inhibition cancel out, causing the apparent zero-state. (D) Example of the 16 synaptic weights of a vector-memory neuron, before and after learning: Before learning (leftmost vector), the synapses all have a weight of (negative) 0.5. After learning, some synapses get depressed toward 0 (inactive), others get reinforced toward negative 1.0. Each of these weights is changed according to the corresponding integrator (CPU4) cell activity at the time of learning.

case, there was no actual nest or feeder object (or associated catchment area) and instead we allowed the search to continue for 10,000 steps.

#### 2.3.2. Memory Re-calibration

We tested the idea of a vector-memory recalibration in simulated open-jaw experiments, by forcing an incongruity between the outbound and the inbound routes similarly to the experiments of Collett et al. (1999) with ants, and Otto (1959) with bees.

In this task, the agent had first to discover a single feeder location by performing a random walk from its nest in an enclosed area to generate the corresponding vector-memory. Subsequently, we let the agent travel again from the nest to the goal location using its vector-memory. Once this was successfully achieved, we simulated a passive displacement by instantaneously changing its coordinates to a novel release location. We then forced the agent's path back to the nest by using wall obstacles disposed in a gutter-like arrangement (see **Figure 4B**). When the agent reached the nest, its integrator would have recorded the forced displacement but not the passive displacement and will therefore not be at the zero-state. The error vector thus encoded was used to make a correction in the vector-memory as described in section 2.2.

The re-calibrated vector-memory was then used in the test task, for N = 100 repetitions. We recorded the paths taken for the averaged re-calibration (efficiency rrec = 0.5), as well as for 10 different values of efficiency. Note that since we only forced an error during the inbound part, this re-calibration becomes a direct way to change the relative weight of the outbound and inbound routes.

#### 2.3.3. Shortcutting

At any point in a vector-memory enabled walk, the agent is driven by the combined effect of the recalled vector-memory and the current home vector. The agent will try to "home" to the location where these are balanced, even if it is forced to take a detour, or has previously moved by itself to another location (e.g., using the vector-memory of a different feeder). Effectively, this constitutes the subtraction of two vectors: one directed from the agent's current location to the nest, and the second directed from the target feeder location toward the nest, so that its behavior follows the vector between their end-points. In other words, the agent should take a direct shortcut to the second food source.

In our shortcutting experiment, the agent first had to discover independently two feeders, by performing two independent random walks (being reset at the nest in-between these walks), storing the two corresponding vector-memories. Then, it used one of these two memories to go back to the associated feeder as described above in the section 2.3.1 experiment. If the first goal is reached, the inhibition from this memory is lifted and the second vector-memory is activated. We evaluated the success rate in reaching the second goal, the path straightness during the shortcut, and the angular error when leaving the first feeder.

As in the section 2.3.1 experiment, we generated a large set of vector-memories, by launching sequentially 1,000 outbound random walks, of length varying between 100 and 10, 000 steps, binned in 20 equally spaced intervals (i.e., 50 independent random walks per length). We then drew N = 1, 000 couples of feeders from this bank so that the straight-line distance between the two feeders ranged between 100 and 2, 000 steps, binned in 20 equally spaced intervals (i.e., 50 independent repetitions for each of the 20 distances bins), while making sure that the Nest - Feeder 1 distance was as uniformly distributed as possible.

#### 2.3.4. Multi-Location Routes

In our multi-location routes experiments, the agent had as a task to take a multi-feeder route, based on a bank of previously stored vector-memories, before going back to the nest.

The order of feeder visits is based on the fact that the distance between the current location and a given memory location can be obtained from the input to the steering cells after inhibition by a specific vector memory (i.e., the subtraction of the 16 synapse weight values from the 16 CPU4 values). The amplitude of the sinusoidal signal across the 16 values directly correlates with the distance between current and memory location. We used an approximation that would be simple to obtain neurally: the sum of the CPU4 activation values after the subtraction of a given vector memory. Note that alternative approximations for the relative distance could be used, such as the value of the cell that is the most active among the 16 cells.

Given k vector-memories, if each is subtracted in turn from the current integrator state rCPU4, then for each we can define a global activity value Score<sup>k</sup> (after clipping the resulting activity between 0 and 1):

$$Score\_k = \sum\_{i=1}^{16} (r\_{CPU4\_{l\_i}} - r\_{VM4\_{k\_i}}) \tag{18}$$

The agent selects the vector-memory generating the smallest Score<sup>k</sup> and sets it as the current vector-memory to drive behavior. However, the scoring process is carried out continuously, so at any time it might change to another vector-memory if its score happens to be lower than the current active one. If the agent reaches a feeder at the vector-memory location, it marks that vector as unavailable for recall for the remainder of the trip. Once no vector-memories are available, it will automatically follow its current PI to go home.

We tested this task in three different feeders arrays: a pentagonal array with 5 feeders where nearest neighbor and the optimal routes are equivalent (Lihoreau et al., 2012b), an array with 6 feeders where the nearest neighbor and the optimal route differ (Lihoreau et al., 2012a) in which real bees were found to select the optimal route, and another array with 10 feeders (Ohashi et al., 2006) but in which real bees were not found to select the optimal route.

To see what sequence of feeder visits would emerge for an agent highly familiar with these arrays, we first allowed the agent to discover and store a vector for each feeder in multiple random walks, repeated for an arbitrary high number of discoveries (at least 100 discoveries per feeder). We then averaged the 100 discoveries to obtain a highly accurate vector-memory for each feeder. Then in the tests, an outward trip corresponds to an agent leaving the nest, exploring or following its memories, and going back to the nest either once all feeders have been found or once a time limit is reached. One trial consists of 50 of these outward trips.

To evaluate performance, we looked at the geometry of the routes the agent realized over 500 repeated trials. The success rate was determined by the number of trials where the agent found all feeders and returned to the nest. Considering only the successful trials, we looked at the sequence of feeder visits, on full routes (occurrence of each possible route connecting all the feeders), as well as at individual feeder-to-feeder moves.

To this end, we only logged the actual visit orders and not the vector-memory recall processes. That is to say, if an agent located on feeder A recalled say, vector-memory of feeder B, but actually missed feeder B and found feeder C instead, we counted this as a path from A to C. Revisits to a same feeder were excluded (as per the bee data, e.g., Lihoreau et al., 2012a,b) by making feeders "disappear" from the agent's detection once they had been visited.

#### 2.3.5. Routes Ontogeny

In order to demonstrate that a route could emerge without necessarily needing the accurate memories used in the previous section, we performed the following experiment on the pentagonal array (Lihoreau et al., 2012b) with a naive agent (without prior knowledge of feeders locations), that gradually learned new food locations through random discovery, while also visiting any locations already learnt:

We here used feeders containing a food amount, and an agent that was assumed to have a crop equal to the sum of all feeders' food (i.e., the agent could only be fully fed after having visited all the feeders). The agent leaves the nest in a naive state, as it does not possess any vector-memory of the feeders in the test environment. The rule is to use vector-memories if any are available, by recalling them using the previously described process, and if no vector-memory is available, perform a random walk until a feeder is found. We also fix a time limit of 10,000 steps, to prevent any saturation that may occur with longer random walks. When a feeder containing food is discovered through random walk, a new vector-memory is created; if a vector-memory is currently active when a feeder is found, this memory is updated (replaced) by the current integrator state. In both cases this updated/newly created vector-memory is not made available to recall until after returning to the nest. As with the traplining experiment, the agent returns to the nest only once all feeders have been visited or when the time limit has been reached.

We observed the change in the duration of the outward trips, the change in total distance walked, and the evolution of the visit sequences. Additionally, we looked at the amount of outward trips needed to visit all the feeders, and to visit all the feeders using the optimal route. Note that once all feeders have been visited, the subsequent trips will be equivalent to those in the section 2.3.4, although memories should gradually become more accurate.

# 3. RESULTS

# 3.1. Memory-Directed Movement

We looked here whether the agent could return from the nest to a location it had reached at the end of a random walk. The agent stored a vector memory at this location, which can be dubbed "feeder location." We tested 20 random walk distances spanning between 100 and 10, 000 steps, with 50 trials per walking distance. To make sure the neurons are not saturating (see **Supplementary Material** section "Saturation" and **Figure S3**), we only used the random walks that ended in a radius of 700 steps from the nest for analysis.

We investigated first the homing performance, by looking whether the agent could home (i.e., reach the nest) from the feeder location. Given an upper limit of 5, 000 steps, the success for the homing task was of 100% (0 out 827 trials failed). We then investigated the ability of the agent to return to the feeder location from the nest, using its vector memory. Given an upper limit of 5, 000 steps, the rate of success in returning to the feeder location was 93.71% (52 out of 827 trials failed). The paths were rather straight (**Figures 2**, **3**), with a straightness index (i.e., beeline/walking distance) of 0.90 for homing and 0.85 for returning to the feeder (which is significantly different for n = 790: paired t-test t = 5.322, p < 0.001). For an analysis of the precision and accuracy of our model in finding the goal, see **Supplementary Material**: Path analysis.

# 3.2. Memory Re-calibration

We aimed here at capturing the ability of insects to recalibrate the outbound vector-memory based on their last inbound run, which we tested by displacing an insect and forcing a homing route that produces a large outbound-inbound discrepancy, as experimentally achieved in ants (Collett et al., 1999). Over 100 subsequent outward trips, the re-calibrated outward paths resemble closely those of real ants. That is, the agent aims at a location that lies in between the two experimental ones: roughly averaging the distance and direction of the previous outbound and inbound paths (**Figure 4C**).

Other studies showed that ants may weight the previous outbound trip more than the inbound trip (Wehner et al., 2002), or even do not recalibrate at all (Wehner and Flatt, 1972). Since the error we introduce is only during the inbound trip, we were able to reproduce these differential weightings of the outbound and inbound trips by varying how much the synaptic weights of

FIGURE 2 | Memory-directed movement. (A) Simplified representation of the CX model with the vector-memory neuron. Layers before (Compass, green; Speed, purple) and after (Steering, blue) the integrator are represented as single nodes for simplicity. Only four integrator neurons (brown) are represented, with their output fibers. The vector-memory neuron (gray) synapses on each of these output fibers with inhibitory connections. These synapses' weights are set during learning according to the activity in the corresponding integrator output fiber, for example by a classic reinforcement process (Reinforcer neuron, red). (B) Examples of memory-directed movements: Large panel, distant Feeder (light green outer circle, Feeder catchment area; green inner circle, Feeder); Inset, Feeder close to the Nest (light red outer circle, Feeder catchment area; red inner circle, Feeder). In both examples, n = 100 individual paths (semi-transparent traces), with 1 more clearly marked. All paths are cut at 5,000 steps if the Feeder is not found.

the vector-memory neuron are modulated by the PI state during re-calibration: from paths aiming at the feeder for weak synaptic change to path aiming at the release location for strong synaptic change overriding the previous memory (**Figure 4D**).

#### 3.3. Shortcutting

We tested whether vector-memories could be used to realize novel shortcuts between two known locations. Here the agent has stored two goals as vector-memories, discovered independently. To test for shortcutting, the agent at the nest recalled the memory of a first feeder and, once arrived at this goal, recalled the memory of the second feeder. We observed whether the

agent was able to strike a direct path between the two feeders (**Figure 5**). Here again, to prevent saturation of the neurons (see **Supplementary Material** section "Saturation" and **Figure S3**) we only considered trials where both feeders were within the radius of 700 steps of the nest. Also, we considered only the agents that successfully reached the first feeder (193 out of 212 individuals).

Given a upper limit of 5, 000 steps, the rate of success in reaching the second feeder from the first feeder was around 89.6% (20 out of 193 individuals failed to reach Feeder 2 from Feeder 1). We carried an analysis of the directional and positional error of the shortcuts displayed by systematically varying the spatial relationship between the nest and the feeders (see "Shortcutting: Error analysis," in **Supplementary Material**).

# 3.4. Multi-Location Routes

data (same random walk).

We tested whether a route could emerge assuming the agent had memorized multiple feeder locations. In this section, the agent already possesses a vector-memory for each feeder location, and the memories do not change over trials. We use a simple heuristic to decide which vector-memory to recall: the agent recalls the memory that yields the weakest overall output activation after subtraction to the current PI state. We tested three different feeder arrays from the bee literature. For each array, we launched 500 independent trials and observed the sequences of feeders visited within a time limit of T = 10, 000 steps (+T<sup>h</sup> = 2, 500 steps for homing).

#### 3.4.1. Positive Array (5 Feeders)

We found that 94.20% (r = 471) of all trials were successful in the sense that all 5 feeders had been visited and the agent went back to the nest before the time limit (**Figure 6B**). There are !5 = 120

FIGURE 4 | Memory re-calibration. (A) Same representation as in Figure 2, with the difference that synapses weights are now modulated by another neuron termed "recalibrator," typically triggered when the agent arrives at the Nest. The weights are modulated in the opposite sign as with the "Reinforcer" neuron of Figure 2. (B–D) Example of the re-calibration effect. (B) Visualization of the training setup. The task is for the agent to leave the Nest (N, Gray circle) and find the Feeder (F, Green circle) by performing a random walk (gray trace). Once the vector-memory of the Feeder is acquired, the agent is reset to the Nest and goes out again on a memory-driven food-ward walk (Orange trace). Then, it is displaced (without any "sensory input") to the Release site (R, Purple circle) and return to the Nest in a home-ward path (Red trace) forced by a gutter (dotted red lines). Feeder, Nest and Release site coordinates were chosen to reproduce the experimental setup in Collett et al. (1999), at scale. Thick gray lines are enclosing walls to enclose the agent for the random walk part. (C) Unconstrained food-ward routes. n = 100 individual examples (semi-transparent traces with one example more clearly marked), guided by the re-calibrated vector memory issued from (A) with an activity of the "recalibrator" neuron of 0.5; an averaged vector appears, replicating the food-ward paths observed by Collett et al. (1999) in ants. (D) Same re-calibration process, but with variable activity levels for the "recalibrator" ranging from 0.0 to 1.0 (increments by 0.05). All paths are cut at 1,000 steps.

possible routes to visit the 5 feeders in this array. We found that, respectively, 77.71% (r = 366) and 15.07% (r = 71) of the trials used the two optimal routes (anti-clockwise and clockwise ; 5, 4, 3, 2, 1 and 1, 2, 3, 4, 5, respectively) ; both cases totalling 92.78% (r = 437) of trials. The sub-optimal nearest-neighbor routes (1, 5, 4, 3, 2 and 5, 1, 2, 3, 4) were used only in 1.49% (r = 7) and 0.64% (r = 3), respectively. Two other routes were used in less than 2% of trials, and 6 other routes were used in less than 1% of trials. The other 108 possible routes to join the 5 feeders were never used (see **Supplementary Table 2** for details).

The overall distribution of direct segments effected between pairs of feeders resembles closely that observed in real bees tested in a similar feeder configuration (**Figure 6B**, **Supplementary Table 1**).

#### 3.4.2. Negative Array (6 Feeders)

In this second array, 94.00% (r = 470) of all trials were successful. There are !6 = 720 possible routes to visit the 6 feeders of this array (**Figure 6C**). Here, only 2.77% (r = 13) of the trials used the optimal route (1, 2, 3, 4, 5, 6). However, we found that 47.23% (r = 222) of the trials used the second to optimal route (1, 2, 4, 3, 5, 6). This route can be described as "suboptimal" in the sense where it is not the shortest, but it is still better than the nearestneighbor route (1, 2, 4, 5, 6, 3), which has been used in 41.28% (r = 194) of the trials. 2 other routes (2, 1, 4, 5, 6, 3 and 2, 1, 4, 3, 5, 6) were used in, respectively, 3.62% (r = 17) and 3.40% (r = 16) of trials, and 4 other routes were used in less than 1% of trials. The other 711 possible routes to visit all 6 feeders were never used (see **Supplementary Table 1** for details).

difference that two distinct vector-memory neurons are available (but only one recalled at a time). (B) Example of shortcutting: An agent walked from the Nest N to a Feeder F1 (light blue outer circle, F1 catchment area ; blue inner circle, F1), under the control of the first vector-memory. Once F1 was reached, the agent recall the second vector-memory and is guided toward the Feeder F2 (light red outer circle, F2 catchment area; red inner circle, F2) by performing a shortcut (vector addition). In both segments, n = 100 individual paths (semi-transparent traces, with 1 more clearly marked). All paths are cut at 5,000 steps if the Feeders are not found.

The overall distribution of direct segments effected between pairs of feeders differs from that observed in bees in this similar feeder configuration. This difference arose mostly because the agents did not perform a direct segment between flowers 2 and 3 as often as the bees did (**Figure 6C**), which we discuss later.

#### 3.4.3. Negative Array (10 Feeders)

In this third array, 95.40% (r = 477) of all trials were successful. There are !10 = 3, 628, 800 possible routes to visit the 10 feeders of this array (**Figure 6D**). The agent explored a much larger number of different routes (371) than in the previous arrays (12 and 9). No preferred route emerged here, the most used route was displayed in only 2.31% of trials. The four most used routes are not optimal in length nor do they correspond to the nearest-neighbor ones (see **Supplementary Table 3** for details), even though they are closer to the latter. The three next preferred route correspond to optimal routes (clockwise and anti-clockwise rotations, either passing through feeder 1 first, or last), and these were used in a total of only 1.05% (r = 5) of trials. 364 other routes have been used in less than 1% of trials each. The other 3,628,429 possible routes have never been used.

This third array appears to be strongly dependent on stochasticity. This is probably due to a combination of two factors: the short distance between feeders yielding stronger directional inaccuracies (**Figure 6C**, and **Supplementary Table 3**); and the similar distance between different feeders options increases the stochasticity of the recall.

#### 3.5. Routes Ontogeny

We used the positive pentagonal array to test whether such efficient multi-location routes could emerge using a naive agent that needs first to discover the different feeders through random walks (**Figure 7A**). Each time the agent discover a feeder, it stores a new vector-memory that will be available for the next trips. The agent was recorded over 50 successive trips. In each trip, the agent would "home" either after a limit of 10,000 steps or if it has visited all the flower locations (i.e., assuming is crop capacity is filled). Over 20 repetitions of such 50 trips' ontogeny, the variation and dynamics resembled that of bees in a similar task. The median amount of number trips needed to find all feeders was 12 (min = 3, max = 20), and the median number of trips needed to realize an optimal route was 13 (min = 5, max = 21). Interestingly, the optimal route did not necessarily emerge as soon as the 5 feeders were discovered, but was achieved within 0 to 2 trips after. This is because some memories can be at first very noisy due to the long random walks that led to their discovery. Across trials, the memories becomes more precise as the agent reaches the feeders more straightforwardly, and the optimal route eventually emerges (**Figure 7A**).

The overall travel distance decreases steadily until reaching a plateau between 20 and 25 trips, close to the shortest straightline distance. Mean traveling speed increases in a similar dynamic, as fewer turns and straighter segments implies faster movements (**Figure 7B**).

# 4. DISCUSSION

Insects such as ants and bees are known to use Path Integration (PI) to return in a straight line to their nest (Müller and Wehner, 1988; Collett and Collett, 2000b; Wehner and Srinivasan, 2003), but also store vector-memories to return to a previously experienced location where they have found food (Wehner et al., 1983; Collett et al., 1999; Wolf and Wehner, 2000). These vector-memories can potentially support additional behaviors such as direct shortcuts between food locations, as shown in previous theoretical models (Cruse and Wehner, 2011). Here we demonstrate that a variety of vector-based navigation behaviors can be obtained from simple extensions to a PI model which follows the anatomical connectivity of the central complex (CX) (Stone et al., 2017).

#### 4.1. Vector-Memories and Novel Shortcuts

The key to the functioning of the model is that, during homing, the steering layer of the CX network continuously compares the distributed encoding of the current heading to a left or right

rotation of the distributed encoding of the PI state (the desired heading). This produces an appropriate left or right turn signal to reduce the difference, resulting in a relatively straight path home, at which point the PI state is balanced. In the extended model presented here, the effect of the PI state on steering can be modulated by inhibition from a vector-memory (**Figure 1B**). The balance point will now be the location where the vector-memory was stored (**Figure 1C**), so the same steering circuit produces a direct path to food (**Figure 2**), as observed in insects (Wehner et al., 1983; Schmid-Hempel and Schmid-Hempel, 1984; Collett et al., 1999; Wolf and Wehner, 2000). Removing the inhibitory effects of memory, once the target location is reached, allows steering by the PI state back home again. Alternatively, switching to inhibition by a different vector-memory produces a direct shortcut from the current location to the next goal (**Figure 5**), as observed in bees (Menzel et al., 2005). As for homing, this steering is robust to any imposed deviation from the intended route (Wehner and Srinivasan, 2003). The way vector-memories are compared to the PI state, and can be selected sequentially to produce shortcuts, is functionally equivalent to former models based on Cartesian vectors (Cruse and Wehner, 2011; Hoinville et al., 2012; Hoinville and Wehner, 2018) but in the present paper it is done with a neurally more plausible ring-neuron representation of vectors.

# 4.2. Dealing With Inaccuracies

Any PI mechanism necessarily accumulates errors (Cheng et al., 1999; Wehner and Srinivasan, 2003), raising the issue of how insects might deal with such errors. If they do not find the goal, whether home or a food source, insect display a systematic search for it (Fourcassié and Traniello, 1994; Merkle and Wehner, 2009; Schultheiss and Cheng, 2012; Wolf et al., 2012). Similarly, the proposed CX model spontaneously results in a search around the expected goal location (**Figure 2**), as in the original model for homing (Stone et al., 2017) and as well as in another model (Hoinville and Wehner, 2018), suggesting that systematic search may not require an additional "search module," as often assumed (Wehner, 2009; Cruse and Wehner, 2011; Wystrach et al., 2013).

The question of PI errors also raises the question of whether and how insects might recalibrate their memories. We introduced two mechanisms by which a vector memory might become more accurate. The first follows from the analysis above—there will be less error in the PI state if the animal reaches a food location on a more direct path from the nest, so

(mean values over 20 repetitions) across 50 foraging bouts: distance, speed and number of feeders discovered. Corresponding insets are examples for one repetition.

increasing precision can be obtained by updating the "active" vector-memory, when the goal is reached, with the current PI value, as we observe in route ontogeny (**Figure 7A**).

There is some evidence in insects of a second mechanism. Manipulating the return path from a food source to the nest can affect the vector-memory (Otto, 1959; Collett et al., 1999; Bolek et al., 2012). We showed how this could be effected in our CX model by allowing the vector memory stored at a goal location (the set of weights) to be adjusted, when the agent has reached home, proportionally to the remaining PI signal, which denotes accumulated errors. This recalibration simply requires the same assumed synaptic connectivity than for learning a vector-memory at the first place (**Figure 4A**). It only implies a second instant in which synaptic weights are altered, rather than an independent PI system for outbound vs. inbound routes. Note that this adjustment could be done simultaneously for all memories either formed or activated on the most recent journey.

In insects, the influence of the homeward path on the next outbound paths varies across experiments (Wehner et al., 2002; Menzel and Greggers, 2015), or sometimes seems non-existent (Wehner and Flatt, 1972). In our model, such variation can be achieved by changing the strength of the synaptic modulation applied during recalibration (**Figure 4**). This effectively results in using different proportions of the PI error when making this adjustment (**Figure 4D**). It remains unclear whether these differences result from differences in species, motivational state, environmental circumstances or individual experience.

Of the "memory neuron" accordingly to the remaining activity of the neurons onto which they synapse. That is, similarly to the way we suggest vector-memory are learnt in the first place, excepted that the synaptic modulation is in the opposite direction, and should happen once the agent has reached home.

#### 4.3. Multi-Feeder Routes

We further extended the shortcut process to explain the development and maintenance of efficient routes between multiple feeders as exhibited by bees (Ohashi et al., 2006; Lihoreau et al., 2012b; Buatois and Lihoreau, 2016). This required two assumptions: 1-the agent needs to select one vector-memory at a time, and 2-a memory becomes unavailable once that location has been visited. We implemented a simple continuous memory selection mechanism, as has been previously proposed (Hoinville et al., 2012). To do so, we used the fact that, in the CX circuit, the inhibition of a target vector-memory onto the PI results in activation levels which amplitude is proportional to the distance to be traveled (**Figure 1C**). At each time step, the current vectormemory recalled can thus be the one that results in the smallest amplitude. Several proxies could be used to approximate this amplitude, but how this is implemented neurally remain to be seen. This produced multi-location routes in our agent that are surprisingly similar to that of bees (**Figure 6**), including the discovery of optimal (shortest possible) routes for some feeder arrays (Lihoreau et al., 2012b), and less optimal routes for other layouts (Ohashi et al., 2006; Woodgate et al., 2017). Alternative hypotheses for memory-selection could exist, but a continuously running winner-take-all mechanism seems parsimonious and readily testable: for example, by enforcing a detour toward a feeder B to a bee on its way to a feeder A and looking for an eventual motivational switch from A to B.

Different ways of storing and selecting vector memories might result in slightly different multi-feeder route outcomes, but the key point is that bees would not need to store, nor compare any additional information (such as path length) about previous journeys to be able to improve their performance over time. Importantly, in this model such multi-feeder routes do emerge, no matter the memory selection mechanism, and without the need to make a comparison of the total traveled distances across successive paths, which was assumed in previous theoretical models (Lihoreau et al., 2012b; Reynolds et al., 2013).

Note that in one of the arrays, the preferred route adopted by our model was not the preferred route of the real bees, but their second preferred one (**Figure 6C**). However, insects do not rely only on vector based strategies, and additional mechanisms, such as the use of terrestrial cues, are likely to modulate the way they follow routes. Spontaneous bias may also influence the shape of a route. For instance, bumblebees have a natural tendency to depart from a flower in the same direction as they arrived (Pyke and Cartar, 1992), which we did not implement here.

Finally, our model could also produce a realistic ontogeny of such multi-feeder routes (note however that we tried here only the regular pentagonal array), given the simple assumption that an agent with no vector-memory available to recall triggers a random walk (**Figure 7A**). In this case vector-memories are gradually added as the agent discovers new flowers. As a consequence, paths become straighter and the revisits order becomes more efficient across successive trips (**Figure 7A**). Interestingly, the ontogeny dynamics of our agents in the pentagon array (**Figure 7B**) resembles that of real bees (see **Supplementary Material** for more details).

#### 4.4. Insights Into Behavior?

Our study thus shows that for direct return to a goal, search around the goal location, shortcuts between goals and efficient route discovery between multiple goals, vector manipulation is a highly parsimonious explanation for observed insect behavior because it appears strongly consistent with the known architecture, and likely computational function, of the CX.

Can our proposed CX implementation however provide predictions about systematic errors in insects, over and above that which has already been provided by canonical PI models (Cheung and Vickerstaff, 2010; Vickerstaff and Cheung, 2010; Cheung, 2014; Hoinville and Wehner, 2018)? We note that the effective PI calculation carried out by our CX circuit model is equivalent to an allocentric Cartesian encoding, and as such, theoretical results concerning the effects of sensory or internal noise on accuracy and precision in return to home or a vector goal derived from mathematical models of this form (Cheung and Vickerstaff, 2010; Cheung, 2014; Hoinville and Wehner, 2018) should apply. This is broadly true for our simulation (see detailed analysis in **Supplementary Material**). For example, we find that directional precision (perhaps counterintuitively) increases with nest-feeder distance, for both inbound and outbound paths, and does not depend on the length of the random walk made before discovery of the feeder, which is consistent with both canonical PI models (Hoinville and Wehner, 2018) and results in ants (Wystrach et al., 2015).

However, we note that observed error effects may be dependent on particular, and somewhat arbitrary, choices in our neural and/or behavioral modeling. For instance, we believe the non-linear activation function of neurons used in the model may explain some of the errors observed, such as an underestimation of distance (see **Supplementary Material**). It is also possible that some of our results are a consequence of (equally arbitrary) parameters in our random walk model (Cheung, 2014). Examination of the consequences of varying these choices would be interesting but is beyond the scope of this paper, which aims to provide a proof-of-principle, rather than provide strong quantitative predictions about animal behavior. However, one general outcome that should hold is that errors for foodward routes should always be higher on average than for homeward routes, as observed here (**Figure 3**), because the control depends on both the current noise in PI and the noise in the vector-memory, from the PI state when it was stored. As the focus of this paper was to show an "in principle" mechanism for vector memory in the insect brain, we leave more detailed examination of how parameter choices in the CX model might affect errors to future work.

# 4.5. Insights Into Neural Circuits

It is of interest to consider whether the neurobiological assumptions made in our model could be verified:

• We modeled vector-memory as simple storage of a copy of the 16 discrete values in the CPU4 layer that represent the home vector at that point in time. We suggest that a vector-memory could be encoded by a single "vector-memory neuron" that sends inhibitory connections to the output of all the integrator neurons (**Figure 2A**). We therefore suggest the existence of such inhibitory neuron projecting to all wedges of the CPU4 outputs or analogous CX layers that would also encode current PI state. Note that similar global inhibitor neurons have been evidenced in drosophila (Kim et al., 2017).


We note that none of these predictions would be trivial to test. However, observing or manipulating the activation of such neural populations in the CX can already be achieved in Drosophila melanogaster (Seelig and Jayaraman, 2015; Kim et al., 2017), and local path integration has also been observed in this animal (Kim and Dickinson, 2017). We further hope that modern genetic tools will soon make this endeavor possible in insects such as bees or ants.

# REFERENCES


# 5. CONCLUSION

The PI model presented in Stone et al. (2017) was mostly based on identified neurons in the CX, whereas the extensions we have proposed here are speculative. Nevertheless, we have provided a proof of concept that direct return to a salient place, search at this locations, vector recalibration, novel shortcuts and even traplining can emerge given minimal additions to the known CX connectivity. A direction for future work would be to consider how such PI navigation system could be integrated with the use of learnt terrestrial cues, which we know affects how bees and ants behave when homing or returning to a known feeding location (Kohler and Wehner, 2005; Wystrach et al., 2011; Mangan and Webb, 2012; Collett et al., 2013), search at the goal (Schultheiss et al., 2013; Wystrach et al., 2013), take shortcuts from novel locations (Menzel et al., 2005; Collett et al., 2007; Wystrach et al., 2012; Narendra et al., 2013; Cheeseman et al., 2014; Cheung et al., 2014), or form traplines between multiple locations (Ohashi et al., 2006; Lihoreau et al., 2012b). The circuitry of the CX is well suited for such an integration of multiple directional cues (Webb and Wystrach, 2016; Collett and Collett, 2018; Hoinville and Wehner, 2018), and as we show here, for a remarkably rich vector-based navigational repertoire.

# AUTHOR CONTRIBUTIONS

FL and TS: model implementation; FL, AW, and BW: manuscript writing; FL, ML, AW, and BW: manuscript reviewing; FL, TS, AW, and BW: conceptual ideas.

# FUNDING

AW and FL: European Research Council, 759817-EMERG-ANT ERC-2017-STG. ML: Agence Nationale de la Recherche, ANR-16-CE02-0002-01.

# ACKNOWLEDGMENTS

We are grateful to Cristian Pasquaretta for his help with the statistics presented in the **Supplementary Material** and Thierry Hoinville for his thorough reviews.

# SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpsyg. 2019.00690/full#supplementary-material


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2019 Le Moël, Stone, Lihoreau, Wystrach and Webb. This is an openaccess article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.