The Role of Learning on Insect and Spider Sexual Behaviors, Sexual Trait Evolution, and Speciation

We review experimental and theoretical evidence that learning in insects and spiders affects the expression of mate preferences and of sexual signals, the evolution of both traits, and ultimately patterns of assortative mating and speciation. Both males and females can modify their sexual preferences and signaling based on previous social interactions or the experience of visual, olfactory, gustatory, or auditory signals. Learning takes place during an early life exposure, previous personal sexual experiences or by observing the choices of others, and it can occur sometimes via very short (a few seconds) exposures to individuals or signals. We briefly review some of the molecular mechanisms that mediate learning in insects, as well as theoretical work that assesses how learning impacts the evolution of insect sexual traits and speciation. We suggest that future research should attempt to provide evidence of the adaptive nature of learning, which remains scarce in insects as well as in vertebrates, and explore further the mechanisms of learning in order to probe into their possible transgenerational inheritance. Future studies should also model how this process might further affect the evolution of sexual traits, and provide a unifying terminology for the underlying mechanisms of learning across diverse life-history contexts.


INTRODUCTION
Sexual behaviors such as the expression of a mate preference or the expression of a sexual signal are often not fixed but can be modified through social experience in both vertebrates and invertebrates. This leads to animals learning a mate preference or learning to display a sexual signal such as a courtship dance or the release of a pheromone blend. Learning sexual behaviors has been accepted for quite some time in mammals and birds, where most research has been conducted (Hebets and Sullivan-Beckers, 2010;Verzijden et al., 2012;Morand-Ferron and Quinn, 2015;Servedio, 2015;Head et al., 2016), including early work by Konrad Lorenz on learned sexual preferences via imprinting. However, learning in sexual selection remained controversial for insects and other arthropods until recently (Dukas, 2006(Dukas, , 2008a. This stems from insects being thought of as having fixed sexual behaviors due to their short lives and few mating opportunities, limiting their possibilities for learning or its likely adaptive value. However, a large number of more recent studies have illustrated that both insects and spiders modify their behavioral sexual interactions upon previous experience. Furthermore, it is now abundantly clear that many species of insects mate multiply, and have complex brain structures allowing short and long term memory of previous experiences that impact their lifetime mating behavior (Dukas, 2006(Dukas, , 2008aChittka and Niven, 2009).
The effect of social experiences, or simply the exposure to a sexual signal such as a pheromone, on the expression of mate preferences and sexual signals has been described in different terms by different authors (Tables 1, 2), from "learning" and "courtship conditioning, " to "mate copying, " to "exposure, " to "premating interaction, " "social learning, " "experience, " "eavesdropping, " "mate preference learning, " and "learned mate recognition, " among a few. This diverse terminology reflects a burgeoning field for insects and spiders over the last decade, as well as the diversity of learning mechanisms that may be at work in these animals. Regardless of terminology (Table 3), what all these cases have in common is a significant change in the expression of mating behaviors that results from a prior social experience or previous exposure to a sexual signal.
Our goals in this review are to: (1) highlight the multiple types of information that insects learn that later result in changes in their mating preferences or in the expression of their sexual signals; (2) propose a systematic categorization of the underlying learning mechanisms, by building on a framework developed in vertebrates ( Table 3); (3) review some of the molecular mechanisms underlying the learning process; (4) review some of the relevant mathematical models, originally applied to vertebrates, that suggest that learning mechanisms have consequences to the evolution of sexual traits and reproductive isolation in insects and spiders. We end by suggesting opportunities for future research in this field.

INSECTS AND SPIDERS LEARN A VARIETY OF SOCIAL INFORMATION AND SIGNALS THAT IMPACT THEIR MATING BEHAVIOR
Learning in the context of mating has been described in a large diversity of spider and insect species (Tables 1, 2). Multiple definitions of learning were used across disciplines, but here, we define learning broadly as being a change in the future sexual behavior of an individual resulting from a previous social experience (Thorpe, 1963;Barron et al., 2015). This definition encompasses all described processes of learning (Table 3), for which the adaptive value of the changed behavior has usually not been demonstrated. Learning often involves changes to mating preferences or to speed of sexual response in females, while it usually involves changes in the level of expression of sexual signals in males. In the majority of investigated cases, and perhaps contrary to what is often assumed, naïve females often do not develop a sexual preference until they are exposed to other members of their species or of closely related species ( Figure 1A). The absence of a naïve preference appears to be especially common for visual sexual signals, which have been the topic of most research. However, when an innate sexual preference is observed, females can also modify it ( Figure 1B) or become more selective (narrow their preference regarding potential mates) ( Figure 1C). In males, learning can lead to changes in the courtship intensity, latency to court, target of courtship, or sometimes the expression of the sexual signals.
Most experimental evidence of learning in altering sexual behavior in insects comes from early exposure of sexually immature adults to the phenotypes of surrounding individuals, what has been called "sexual imprinting" in vertebrates. However, sexual behaviors can also vary after the observation of the interaction between other mating individuals, such as in cases of mate-choice copying (e.g., Mery et al., 2009) or imitation of sexual signaling (e.g., Clark et al., 2012Clark et al., , 2015, or from previous sexual interactions with or without mating (Tables 1-3). Most studies do not demonstrate that male or female behavioral shifts have an adaptive value, but such value is often assumed. Below, we illustrate this growing experimental evidence of learning in arthropods organized by the type of learned information: social information with unidentified individual signals (when the full phenotype of the interacting individuals is provided to the "learner"), or of specific signals such as olfactory, tactile, visual, and acoustic including vibratory signals (see Tables 1, 2 for a more complete overview, Figure 2).

Social Exposure Without Clearly Identified Learned Signals
Authors report that a social experience, with or without mating, affects subsequent sexual interactions of focal individuals. In most of these studies, adults are exposed to the full phenotype of other individuals where the specific assessed and learned signal(s) are not clearly identified. For example, virgin female crickets reared in isolation approach and contact males less frequently than virgin females reared in a group (Tinghitella, 2014). Being housed with groups of males or females induces fruitless knockout Drosophila males, which have lost their ability to court, to recover their courtship behavior, and wild type males to reduce their same-sex sexual behaviors (Bailey et al., 2013;Pan and Baker, 2014). Burrow-digging spider males enlarged their burrows upon rejections by females, which increases their chances of mating, as females prefer larger burrows (Carballo et al., 2017).
The mating status or novelty of the interacting individuals appears to be particularly important in modifying a focal individual's subsequent behavior. For instance, naive female crickets and spiders are more likely to mate, copulate more quickly or cannibalize fewer males than their mated counterparts (Johnson, 2005;Wilder and Rypstra, 2008;Judge et al., 2010). Male fruit flies learn to focus their courtship toward receptive conspecific females, and out-compete sexually inexperienced males, based on previous copulations (Saleem et al., 2014), or based on rejections by mated or virgin females (Dukas, 2005;Ejima et al., 2005;Griffith and Ejima, 2009). These males also learn from rejections from heterospecific females (Dukas, 2004(Dukas, , 2006(Dukas, , 2008b(Dukas, , 2009Kujtan and Dukas, 2009;Dukas and Dukas, 2012;Dukas and Baxter, 2014) or immature males (Gailey et al., 1982;McRobert and Tompkins, 1988;Bretman et al., 2010). Learning to quickly discriminate receptive from unreceptive individuals is likely adaptive as males can reduce the costs of unsuccessful courting and mating with an unreceptive female or  Males rejected by immature males, and immature males receiving courtship by other males, increased their time courting virgin females, increased their copulation duration and reduced their mating latency (compared to solitary males). Gailey et al., 1982McRobert and Tompkins, 1988Bretman et al., 2010Dukas, 2010 Fruit fly Drosophila melanogaster

Visual learning
Visual. Virgin males trained by mating with a female with a specific eye color for 1.5 h. Number of male courtships and male choice measured in a two-choice assay 24 h after training.
Males preferred females having the same eye color as the one they were trained with.

Verzijden et al., 2015
Fruit fly Drosophila melanogaster Preference for phenotypic novelty Full phenotype. Training: mating of 5-day-old males with a random female. Male preference and courtship quantity measured in a two-choice assay with live females immediately after training, or with decapitated females 30 min after training (choice between the female the male previously mated with, and a novel female).
Males courted the novel female more. Olfaction was involved as Orco mutants didn't discriminate between the two types of females. Tan et al., 2013 Fruit fly Drosophila melanogaster Auditory plasticity Acoustic. Females trained with conspecific or heterospecific songs during 6 days from emergence. Male chaining behavior (proxy for sexual arousal) measured 1 day later. Sounds broadcasted with loudspeakers.
Males selectively reduced the response to heterospecific male songs (songs are part of the male courtship ritual) only after having experienced conspecific songs. Li et al., 2018 Fruit fly Drosophila melanogaster

Prior information and social experience
Visual. Training: 6-days-old adult males placed with groups of 6-day-old big (high quality) vs. small (low quality) females for 30 min. Mate choice 1 h after exposure in a two-choice assay.
Males exposed to large females courted large females more often, whereas males exposed and mated to small females courted them more often.  Presence or absence of male-male courtship in a no choice (2 paired males) and two-choice (between a male and a female) assay at 9-days-old was measured.
Males reared in isolation showed more same sex behaviors than males reared in groups of males or females. Avoidance learning and mating experience impacted the expression of same sex behavior. Bailey et al., 2013 Bee fly Megapalpus capensis

Learning
Visual. Virgin males exposed to deceptive floral forms producing strong mating behavior: Males were released in a pollinator cage with deceptive flowers where their feeding and sexual behaviors when visiting flower forms was recorded for 10 min. Males were also caught from areas where the sexually deceptive flower was absent (inexperienced males) or from areas where it was present experienced males. Males released again in the pollinator cage with deceptive flowers after 10 min. The same behaviors were recorded.
Males showed less mating behavior toward the fly-mimicking spots of the deceptive flower form during their second exposure, or when they were caught in areas with deceptive flowers present.

Learning
Visual.
Bees exposed to various deceptive or not deceptive flower patterns, and being rewarded with sugar while choosing the right pattern.
Landing near a pattern, approaching or turning back from the pattern were recorded while several choices proposed Labellum patterns of different flowers can be reliably learnt; but patterns of flowers from the same inflorescence cannot be discriminated in a lab set-up. Bees can probably learn to recognize the deceptive flower patterns in the field.

Learning
Visual. Wasps landing on a presented deceptive flower and contacting its column were caught, marked and released. Re-capture experiment was done 4 days after, with the same display. Recapture rates within a day and within a week were calculated. Copulation attempts were also recorded with successive wasp visits on the presented flower.
The rate of wasps that visit the deceptive flower decreased highly within a day and a week, but the attempted copulations decreased with successive visits too, suggesting that individuals were learning to recognize the flower.

Habituation
Olfactory. Exposure of males to female odors (impregnated on a filter paper) Subsequent approaches and contact with females and with papers impregnated with females' odors were measured, Male exposure to specific female odors induces a reduction of sexual contact frequency with females (compared to before exposure). Habituation lasted over 24 h, and was proposed to promote outbreeding and prevent the cost of mating with unreceptive females. Barrows, 1975 Sweat bee Lasioglossum zephyrum Learning or habituation Olfactory. Training by housing one adult male with one adult female together, allowing copulation attempts for 10 min First female replaced by a second one whose genetic relationship to the first is known. Number of male copulation attempts for 2.5 min was recorded, Males rejected females if they were genetically close to the one they were previously exposed to. Males learned to recognize the female odor. The learning promotes outbreeding. Smith, 1983 Sweat bee Lasioglossum zephyrum Learning Olfactory. Females sampled from different nests and frozen to create pacifier females. Some females had their odors removed. Number of males approaching and contacting the pacifier females in every different nest for 15 min was measured. Measurements were repeated at every nest, several times.
Males contacted females more during the first than during the second presentation. Nestmates were less attractive than non-nestmates (probably due to them being closely related). Wcislo, 1987 (Continued) Frontiers in Ecology and Evolution | www.frontiersin.org Vibrations and sounds of the developing parasitoid (14-16 days post-parasitism) when contacted by a male parasitoid were also recorded.
Males revisited the location of hosts containing developing parasitoid in the field and in laboratory experiments. Males stayed with the host when the emergence of the parasitoid (a prospective mate) was imminent. Male contact and proximity with a parasitized pupa induced vibrations of the developing parasitoid, which were used as cues to localize mates. Danci et al., 2013Danci et al., 2014 Asian citrus psyllid Diaphorina citri Learning Olfactory.
Training: 1-to 2-week-old adult males mated from a colony of mixed individuals, or mated with a female carrying a food odor for 24 h. Virgin males were from a male colony only.
Male attraction to female odor tested in Y olfactometer for 5 min.
Male attraction to female odors significantly increased after a mating experience. Male attraction to the food odor alone did not. Stockton et al., 2017 African field cricket Gryllus bimaculatus

Mating
Full phenotype. Sequential exposure of 6-to 10-day-old post-adult molt males to large and small females (exposure to large, then small, then large and small; or exposure to small then large, then small and large). Male latency to court and eagerness to copulate was measured.
Naive males courted both types of females equally, but mated males courted large females more. Bateman and Fleming, 2006 Pacific field cricket Teleogryllus oceanicus Juvenile social experience Acoustic. Juvenile males (stage preceding their penultimate instar) exposed to a mix of six songs. Attraction (time spent near the speaker) of 6and 13-day-old adult males to the other male's song was measured.
Males reared in silence exhibited more satellite behavior (they spend more time near the speaker) than males reared in a rich acoustic environment. Males from the silent population of Kauai increased their encounter rate with females responding to other callers.
Males reared from late instar nymphs in silence, or exposed to songs that had increasing syllable numbers, and to a mix of song made of all the syllable numbers for 6 h a day. At 6-8 days post-adult molt, each male had its number of songs and number of syllable per song recorded.
Male call rates as adults decreased with exposure as juvenile to various male calls compared to males reared in silence. Males invested more in attracting a mate in the absence of competition.  Wolf spider Schizocosa ocreata Eavesdropping Visual and olfactory. Sexually mature adult male (laboratory-reared or field-collected) exposed to video playback of a courting male stimulus for 3 min in combination with female silk cues or with the video only; or males paired in an arena with another courting male for 5 min. Trials videotaped, and focal males scored for frequency of courtship tapping displays (after 3 days of conditioning).
Males copied the courtship dance of other males, from videos or from real males. Field-collected males copied more courtship sequences than lab-reared males. Males also learned to associate the courtship with female cues, as they showed higher levels of courtship when female odors were present. Courtship was also increased when male density was higher. Clark et al., 2012Clark et al., 2015 Wolf spider Hogna helluo

Exposure
Full phenotype, visual, and olfactory. Exposure without mating: adult females paired with males, no mating allowed, but visual and olfactory cues available; or males kept individually without exposure. Exposure done on days 1 and 3, followed by mating trials on day 4. Exposure with mating: females mated with males, and mating trial done 1 week later. Mating trial: females paired with a male, and mating number, latency and copulation duration were measured.
Males exposed to female cues were less likely to mate with the proposed female compared to control males. Authors propose that exposed males likely perceived a high availability of females and took fewer risks (they became choosier). Previously mated males increased their subsequent mating frequency and had lower risks of cannibalism. These males might be perceived as higher quality mates. Wilder and Rypstra, 2008 (Continued) Frontiers in Ecology and Evolution | www.frontiersin.org Experience and learning Olfactory, tactile and full phenotype. Adult males exposed to the silk of a non-virgin female for 30 min a day over 5 consecutive days; or adult males non-exposed; or, one virgin male paired with one virgin female for 1 h. 24 h after exposure or mating, one male and one non-virgin but sexually receptive female were paired for 30 min. Record of courtship latency, duration, number of copulation attempts and copulation latency.
Males experienced with female silk courted non-virgin females for a longer duration than inexperienced males. Males that were already mated rarely courted a second female and never copulated with the second female in this experiment.

Experience of female feedback
Visual and seismic.
Males trained on a substrate transmitting vibratory cues, or on a hard substrate that didn't transmit the cues. Pre-training evaluated male's baseline substrate use (5 min); followed by a training where males received feedback from a puppet female (they received visual and seismic cues from the puppet, or seismic cues only, or visual cues only). Control males received no feedback.
Male subsequent courtship behavior (number and location of body bounce) toward female's odors measured during 5 min right after the training.
Males that experienced feedback from the female puppets increased their number of seismic signals and chose a better substrate for their vibrations. These males that adjusted their courtship were more likely to copulate, reached copulation more quickly, and were less likely to be cannibalized than males that didn't change their signal. This is one of the few studies linking the learning process to fitness effects.

Experience rejection by females
Full phenotype. Naïve adult males (with burrows already built) non-exposed or exposed to virgin or mated females. If male was rejected by the female, the male was left for 48 h when he could modify his burrow, and was paired with the same virgin or the same mated female. Burrow dimensions were measured.
Males exposed to females that reject them learn to enlarge their burrows which become longer than unexposed males, and therefore increase their future chances of mating (females prefer large burrows). Here, male learning directly improves his reproductive success.

Bicyclus anynana
Biased learning Visual. Newly emerged males exposed to females with either zero or two dorsal hindwing spots for 3 h. 2-day-old male preference measured in a two-choice assay with a zero and two dorsal hindwing spot females.
Males exposed to 0-spot females (on the hindwing) for 3 h learned to prefer them in subsequent mating trials, while naïve males and those exposed to 2-spot females mated randomly.

Habituation
Olfactory. Pre-exposure: 1-day-old males exposed to the female's main sex pheromone component. Males' locomotor activity measured before, during, and after the pre-exposure to the component. Males tested for their locomotor activity again 24 h later upon exposure to the female sex pheromone blend.
Males increased their activity during the pre-exposure to the main pheromone component, compared to before and after pre-exposure. The subsequent trial with the complete sex pheromone blend showed lower male activity peak than during the pre-exposure to one pheromone component.

Suckling et al., 2018
Noctuid moth Copitarsia decolora Pre-exposure Olfactory. 4-day old virgin males exposed to 3 female equivalents of the sex pheromone gland extracts for 10 seconds. Control males were not exposed. Male activation, flying toward the source and landing on the odor source latencies measured 2, 24 and 48 h after exposure in a wind tunnel with the sex pheromone gland extracts as attractor.
Pre-exposure induced an increase in latency of activation and landing at the odor source, so it induced a short-term (2 h) inhibition of the response to the olfactory stimulus.

Sensitization
Olfactory. 2-to 4-day-old naïve virgin males shortly (∼10-30 sec) pre-exposed to 1 female equivalent of sex pheromone extract, or to the main component. Control males were not exposed.
From 15 min to 51 h after exposure, male behaviors tested in a wind tunnel where they were recorded for their orientation and movement toward the main pheromone component or gland extracts.
Increased activation and attraction to the odor source in pre-exposed males compared to control males even 51 h after pre-exposure. This behavior is attributed to a sensitization of the primary olfactory centers. Anderson et al., 2003Silvegren et al., 2005Anderson et al., 2007 Frontiers in Ecology and Evolution | www.frontiersin.org Pre-exposure of naïve males to pulsed bat-like sounds (predator sounds). Control males were not exposed. Male behavior tested in a wind tunnel where mare orientation and movement toward the sex pheromones were recorded.
Increased behavioral and neuronal sensitivity of the male to female sex-pheromones upon exposure to predator sounds. Moths were thus capable of integrating bimodal sensory information.

Experience
Olfactory. Laboratory set-up: males reared as larvae on cotton leaves or cotton leaves or artificial diet until pupation followed by measurement of 3-4-day-old male attraction to female sex pheromones associated with plant odors in a two-choice wind tunnel assay in the laboratory.
Field work: larvae reared on cotton or on alfalfa, then 8-day-old pupae were transferred to the border between cotton and alfalfa fields. Pheromone traps baited with the major component of the female sex pheromone blend were placed in the fields. Number of males captured in each field was recorded over 10 days.
Males were more attracted to female sex pheromones combined with the odor of the host plant species they experienced as larvae than to sex pheromones combined with odor from the plant they had not experienced. By selecting a female on a plant of the same species that supported his own larval development, the male controls the plant quality for his offspring. Anderson et al., 2013 Rove beetle Aleochara curtula Habituation and learning Full phenotype, olfactory, gustatory and tactile.
Males presented with a mated female pacifier five times in succession. Some female pacifier cuticular hydrocarbons were manipulated to make them either similar or different in scent and presented one after the other. First presentations were followed by five additional presentations of either another mated or a virgin female pacifier. Males grasping responses on females and pacifier were measured at each presentation.
Males learned to recognize the cuticular hydrocarbons of the females they mated with and reduced subsequent mounting attempts with these females.

Schlechter-Helas et al., 2012
Familiar bluet damselfly Enallagma civile Learned mate recognition Visual. There are differently colored female morphs in the species. Sexually mature males caught in the field, marked, and released inside an enclosure with equal number of andromorphic or heteromorphic females. Interactions happened for 2 days. Males presented to live tethered females of different morphs, at different times of the day, and male behavior was recorded: no reaction, approach, grab and tandem formation (copulation). Males were also tested in a two-choice assay with females of different morphs.
Sexually mature males learned to prefer the color morph (including the andromorphic females) they have been previously exposed to (interacting and mating). Naïve males didn't have a preference. They learned the most common morph encountered in the field. Miller and Fincke, 1999Miller and Fincke, 2004Fincke et al., 2007 Familiar bluet damselflies Enallagma aspercum E. civile

Experience
Full phenotype and visual. Female morph frequencies were tallied during the morning and the afternoon in both species.
Male sexual behavior (approach, grab, tandem or takeover of another mating) measured when presented to tethered females of the different species in the morning or in the afternoon, when frequencies of each morph were different.
Males of both species reduced their heterospecific sexual interactions with females in the afternoon, after having interacted with them. The male's sexual response toward female morphs of both conand heterospecifics varied over the course of a day in response to changes in the density of female morphs.
There are differently colored female morphs in the species. Successive exposure of males to different female morphs or to other males during 2 days. Measure of male preference after each exposure by two-choice assay with two female morphs proposed to the male for mating.
Males preferred the most recently encountered female morph. Males changed their preference after each successive encounter based on the most common morph. Male chose their mate in a frequency dependent way. Males exposed to males also formed a majority of tandems (copulation attempts) with other males, but when these males were housed again with a mix of males and females, they preferred the more abundant female morph.
van Gossum et al., 2001van Gossum et al., 2005 Frontiers in Ecology and Evolution | www.frontiersin.org Visual. Lab assay: 5-to 9-day-old sexually mature virgin adult males paired with a single sexually mature female, either an andromorph or a gynomorph, in the morning for 4 h. Control males were reared isolated. Fieldwork: Female morph frequency tallied in the field, in the morning and in the afternoon. Male preference tested in a two-choice assay with both female morphs (female dummies) in the afternoon in the lab, and in the morning and afternoon in the field.
Virgin males had no innate mating preference for a female morph, but males that experienced copulation significantly preferred the same morph they copulated with. In the morning, males didn't discriminate between female morphs, but they preferred the dominant female morph in the afternoon.
Males changed their preference based on the most common morph present in the field.
Takahashi and Watanabe, 2010a Takahashi and Watanabe, 2010b Marsh bluetailed damselfly Ischnura senegalensis Learning Visual learning. Virgin male paired with a sexually mature female gynomorph or andromorph in the morning until mating. Control males were reared isolated. Immature and mature gynomorphs have brightness differences not shown by andromorphs. 6-9 day old sexually mature males submitted to two-choice assays with immobilized sexually immature (1-to 3-day-old) and sexually mature females (5-to 9-day-old) of both morphs.
Virgin males didn't have a preference between an immature and mature female, but males that had experienced copulation with gynomorphs preferred sexually mature gynomorphs to sexually immature ones. Males experienced with andromorphs didn't discriminate. Males might be learning the color differences between mature and immature gynomorphs.
Takahashi and Watanabe, 2011 Treehopper Enchenopa binotata species complex Social experience Acoustic. Adult males exposed 1 h per day for 14 days to a range of male calling frequencies corresponding to con-and heterospecific signals, to a mix of signal frequencies, or to silence. Males also exposed to a mix of male calls and to different female signal responses. Male's signals the day after their last exposure (minimum 18 h) were recorded.
Males exposed to a range of conspecific competitors (mimicked by signals with average frequencies) signaled faster (higher rate) than non-exposed males or males exposed to unattractive signals. Also, males called for longer time when they were exposed to females that responded to their preferred male call frequency. Rebar and Rodriguez, 2016 a male. Females in many species also prefer a new male over their previous mate. For instance, in crickets (Bateman, 1998), in moths (Xu and Wang, 2009;Li et al., 2014), in hide beetles (Archer and Elgar, 1999), or in female Drosophila melanogaster, a simple exposure without mating is enough to trigger a similar preference for a new male (Odeen and Moray, 2008;Loyau et al., 2012). This type of learning might also be adaptive as by rejecting the males that they saw copulating, females could reduce the costs of mating with semen-limited males (Loyau et al., 2012).

Visual Signals
Multiple studies have shown that naïve individuals have no innate genetic mate preferences for particular visual signals but develop these through learning. In many cases, naïve males direct their courtship toward a wide range of females, while inexperienced females display no preference for a specific male visual trait. For instance, male fruit flies, who initially court both large and small females with equal vigor, will preferentially court one of these female types if previously exposed or mated to them (Balaban-Feld and Valone, 2017). In wolf spiders, juvenile or adult exposure to male tibia types is necessary to limit the female preference to a specific leg tuft size or color (Hebets, 2003(Hebets, , 2007Rutledge et al., 2010;Uetz, 2015, 2016a,b). Male damselflies learn to prefer the female color morphs they previously interacted with Fincke, 1999, 2004;van Gossum et al., 2001;Fincke et al., 2007;Watanabe, 2010b, 2011). Experienced female damselflies learn to reject heterospecific males by recognizing their wing patches (Svensson et al., 2010(Svensson et al., , 2014Verzijden and Svensson, 2016). The acquisition of a preference for visual traits has also been reported in fruit flies [eye color, (Verzijden et al., 2015)], butterflies [hindwing ornamentation number, (Westerman et al., 2014)], or crickets [size, (Bateman et al., 2001)] (Figure 2). Recent studies on matechoice copying showed that virgin individuals tend to prefer male phenotypes with similar color type and ornamentation as the mate choice of another conspecific, e.g., in spiders, (Fowler-Finn et al., 2015) and fruit flies, (Mery et al., 2009;Nöbel et al., 2018a). These cases illustrate that insects can generalize socially learned public information for choosing a mate. Finally, male spiders can also copy the leg-tapping dance of other courting males to increase their likelihood of seducing a female (Clark et al., 2012(Clark et al., , 2015; Figure 2). These males reach copulation more quickly and are less likely to be cannibalized (Sullivan-Beckers and Hebets, 2011Hebets, , 2014. In the few documented cases of insects displaying innate (naïve) visual signal preferences, learning can be context dependent, and sometimes may override, or even reverse these preferences. For instance, female Bicyclus anynana butterflies can switch their naive preference from males with two UVreflective spots on their forewings to four spots if they are  Females preferred to mate with the same male color type they previously observed mating. Naïve females had no preference. Mate copying is stronger when the demonstration happened sequentially, and in higher atmospheric pressure (better weather conditions). Mery et al., 2009Germain et al., 2016Dagaeff et al., 2016Nöbel et al., 2018b Fruit fly Drosophila melanogaster Mate copying avoidance Visual. Training: 3-day-old female observations of both a mating pair and of a male being rejected by a female for 45 min.
Measure of female mate choice in a two-choice assay just after the observation.
Females preferred to avoid the specific male they saw copulating before and mated with the other male. By rejecting the males they saw copulating, females could reduce the costs of mating with semen-limited males. Training with conspecific male songs reduced female acceptance of heterospecific songs (copulation acceptance).

Exposure
Full phenotype and visual. Adult female sequential exposure and mating to males of different sizes (every ∼30 min). Female acceptance or rejection, mating latency and spermatophore retention recorded for each male presentation.
Females previously mated with males of any size favored big males in subsequent matings, compared to virgin females who didn't discriminate between male sizes. This process may help a female ensure that she is choosing the most attractive mate available in her social environment.
More than 14-day-old virgin adult females sequentially exposed to songs, twice per day for 3 days. Female attraction to the speaker was measured.
Female responsiveness (attraction to a preferred song) decreased with exposure to a series of songs.

Shaw and Herlihy, 2000
Pacific field cricket Teleogryllus oceanicus Social experience Acoustic. 5-to 8-day-old virgin females exposed to preferred and non-preferred male courtship songs during mating trials with silenced males (< than 10 min). 24 h later, female latency to accept copulation and latency to reject spermatophore after copulation were measured.
Females mated with males with preferred (= attractive) songs mounted subsequent males more slowly and had lower subsequent sperm retention durations than females mated with males with non-preferred songs. Rebar et al., 2011 Variable field cricket Gryllus lineaticeps Acoustic experience Acoustic. 7-to 30-day-old virgin females exposed to sequences of songs with different chirp rates, naive females preferring high chirp rate songs. Female distance to the speaker was measured as a song attractiveness proxy in three sequential trials separated by a 20 min resting period.
Females exposed to a sequence of attractive (high chirp rate) and unattractive songs were more attracted to attractive songs than females exposed to unattractive songs only.

Wagner et al., 2001
House cricket Acheta domesticus Exposure to conspecifics Full phenotype. Adult females reared in isolation or in low and high density groups (with equal numbers of males and females). Courtship trials conducted when females were ∼10-and ∼32-day-old. Frequency of female approaching males and number of mounts were measured.
Females reared in groups approached and contacted males more frequently than females reared in isolation.
Tinghitella, 2014 Female nymphs within 24 h of hatching exposed to different song intercall durations, mimicking different male densities and calling rates. Upon maturity, females isolated for 10 days. Then, female choice measured in a two-choice assay, each female used in 6 consecutive choice trials.
Females reared in a "middle calling song rate" acoustic environment decrease their response latency toward future calls, compared to females reared in low or high calling rate acoustic environments. Females responded more quickly in later trials. Kasumovic et al., 2012 Pacific field cricket Teleogryllus oceanicus Social learning Acoustic. 5 min exposure of 6-to 7-day-old females to different calling song models (preferred and non-preferred by the females).
Female subsequent attraction to the speaker playing a reference song measured after 30 s rest.
Females that experienced preferred songs had lower attraction to the reference song compared to females exposed to other songs. Females experienced with less-preferred songs showed higher attraction to the reference song. Bailey and Zuk, 2009 Pacific and black field crickets T. oceanicus T. commodus

Social learning
Acoustic. Juvenile females from 4th instar reared in silence or exposed to a mix of male calling songs that mimicked a natural setting. 6-day-old mated females also exposed to songs and silence as adults.
Female phonotaxis -attraction to the speaker playing different calling songs -tested 6-10 days post eclosion, each with 16-24 h of silence prior to testing. Female phonotaxis also tested in a two-choice assay between con-or heterospecific songs.
Adult or juvenile females reared in silence (mimicking the Hawaiian Kauai silent population) were more responsive (more attracted) to future callings than females reared in a mix of songs. Females seem to accommodate the loss of sexual signal in this population. Bailey and Zuk, 2008Bailey and Macleod, 2014Swanger and Zuk, 2015 Pacific field cricket Teleogryllus oceanicus Social flexibility Acoustic and indirect genetic effects on female preferences. Females from 5 different populations exposed to silence or to a mix of male calling songs that mimicked a wild setting with a high density of calling males. 6-10 days post eclosion female placed with one silent male to estimate females' latency (=choosiness) to mount.
Female's previous experience altered their choosiness depending on studied populations. In Hilo populations, female choosiness decreased with song experience, while in the Samoa and Oahu populations, experience didn't affect female behavior, and contemporary Kauai females (from a silent population) were choosier after exposure to male songs, whereas ancestral Kauai females were less choosy after exposure to male songs. Indirect genetic effects affect mate choice and are likely to have an evolutionary impact of the populations.
Rearing of females from late instar nymphs to 13-day post-adult molt in silence, or exposed to songs that have increasing syllable numbers, and to a mix of songs made of all the syllable numbers for 6 h a day. After 24 h, each female tested with several acoustic stimuli from 1 to 10 syllables. Females' movements toward the sound source recorded with a locomotion compensator sphere. Construction of the female mate preference function to measure her selectivity (see Figure 1).
Females from every acoustic environment preferred songs with high syllable rate. The females exposed to a mix of songs were more selective than the other females.

Mating
Full phenotype. Pairing of adult virgin males and females until copulation. 12 h later, mate choice between the same male and a different male was recorded.
Females preferred to re-mate with a novel male rather than with a male they already mated with. They gain benefits from multiple mating.

Exposure and subadult experience
Visual. Subadult females (penultimate stage before molting into adult) exposed to courting males with the tibia and patella painted in brown or in black for 30 min per day until final molt, control were non-exposed females. 11-to 20-day-old females paired with a male of one of the two phenotypes, and occurrence of copulation, latency to copulation, and occurrence of cannibalism, during 30 min trials was recorded.
Exposed females were more likely mate with a male having the same color as the one they were exposed to. Females were also more likely to cannibalize a male with non-familiar color. Control naïve females had no bias for either morph.

Learned signal and methodology Altered behavior References
Wolf spider Schizocosa rovneri

Exposure and subadult experience
Visual. Subadult (penultimate instar) females exposed to courting adult conspecific males with forelegs painted with black or transparent nail polish, or with a black or transparent nail polish dot on the cephalothorax (odor-control males). 7-21 days following adult molt, female receptivity or willingness to mate with males of the different phenotypes was measured.
Females exposed to black foreleg males were later less receptive to these specific males, and less likely to mate with them, than to the males painted with transparent nail polish, and to control males. Females exposed to legs painted with transparent nail polish preferred males with black paint over other males. Unexposed females showed no innate preference. Difference between these results and Hebet's earlier work (2003-above) might be explained by differences in experimental set-up (e.g., the type of paint used, as the paint odor might impact the female's response toward the visual cue).

Exposure and subadult experience
Visual. Subadult females (last instar before adult molt) exposed to courting brushlegged males (sensu S. ocreata) or non-ornamented males (sensu S. rovneri) every 2-3 days during 30 min each. Non exposed females are control.
Female paired with one of the male forms 13-24 days after their adult molt. Occurrence of copulation, latency to copulation, and occurrence of cannibalism, during 30 min trials was recorded.
Experienced females preferred brushlegged males (S. ocreata), regardless of the male they were exposed to, whereas inexperienced females showed no mating preference (between ornamented and non-ornamented males sensu S. rovneri).

Hebets and Vink, 2007
Wolf spiders Schizocosa ocreata S. rovneri (allopatric species) Prior juvenile experience Visual and seismic. Subadult females (last instar before adult molt) exposed to conspecific or heterospecific courting males every day during 30 min each. Control females were not exposed. Contact between individuals not allowed (they shared visual and seismic cues only). Females paired with one of the males 7-14 days after their adult molt. Female receptivity behavior and occurrence of female aggression or cannibalism was recorded.
No effect of pre-exposure: exposed or unexposed females preferred their respective male species. More exposure to conspecifics reduced females' S. ocreata aggressiveness toward conspecifics. Exposure to heterospecific male courtship did not affect aggression in S. ocreata. Exposure didn't affect S. rovneri aggressiveness.

Socially cued anticipatory plasticity
Visual. Subadult females (last instar before adult molt) exposed to videos of courting males with small or average, or large tibia tufts (foreleg bristles) size, or a mixture of sizes, until adult molt. Exposure done at various frequencies.
Female receptivity behavior for courting males (on videos) tested at 10-15 days after their adult molt in no-choice (males with large or small tufts) and two-choice assays (large and small tufts).
Females exposed to average leg tuft size, a mix of sizes, and large tufts were more receptive to future courtship of large-tuft males over small-tuft males.
Females exposed to small-tuft males were later more receptive to future courtship of small-tuft males than large-tuft males (when exposed as juvenile or as adults). Naïves didn't have a preference.

Stoffer and Uetz, 2015 Stoffer and Uetz, 2016a
Wolf spider Schizocosa ocreata Adult social experience Visual. to-6-days-old adult females exposed to videos of courting of males with small or large tibia tufts. Frequency and number of exposure varied. Female receptivity behavior toward video of 7-day-old courting males in a two-choice assay (large and small tufts). Females pre exposed to small-tuft males displayed more receptivity toward small-tuft males, while those exposed to large-tuft males displayed more receptivity toward large-tuft males.

Courtship modality experience
Visual and seismic. Subadult females (last instar before adult molt) exposed to videos of courting males with an average leg tuft size, or exposed to a vibratory playback (mimicked courtship seismic signal), or exposed to both. Control females were not exposed to any signal. Exposure ended at the final molt to adult stage. Female's latency to approach the source of each signal, and of the receptivity behaviors at 10-15 days after their adult molt was measured in no-choice (one female was tested with the visual signal only, then the seismic signal only and then both signals) and two-choice assays (visual and seismic).
Female juvenile experience didn't affect their subsequent latency to approach the signal, as they all approached the courtship signals faster when playback included visual signals (visual only and multimodal). Females displayed more receptivity toward the unimodal signal modality they were exposed to, but they preferred the multimodal signal above all other ones, regardless of exposure (in the no choice assay).

Mate choice copying and learning
Visual and seismic. Virgin females exposed visually to another female choosing between a male with a tibia tuft and a non-ornamented male. The female was exposed to the courtship and copulation happening to the other female or to the copulation only. Control females were not exposed. The exposed female could perceive the visual and vibratory signals of the interacting trio, but could not physically interact with them. Female preference for the ornamented or non-ornamented male tested in a two-choice assay just after the exposure.
Exposed females matched the mate choice (ornamented vs. non-ornamented males) of the females they observed being courted and mated.

Fowler-Finn et al., 2015
Wolf spider Hogna helluo Exposure Full phenotype, visual, and olfactory. Exposure without mating: adult females paired with males, no mating allowed, but visual and olfactory cues available; or females kept individually without exposure. Exposure done on days 1 and 3, followed by mating trials on day 4. Exposure with mating: females mated with males, and mating trial done 1 week later. Mating trial: females paired with a male, and mating latency, copulation duration and female aggression frequency were measured.
Mated females had lower probability to mate than virgin females. Females exposed to visual and olfactory cues took less time to mate than non-exposed females. Previously mated females were significantly more likely to engage in precopulatory cannibalism than virgin females. Virgin females exposed to males as juveniles showed more pre-copulatory attacks than females exposed to another female, or than non-exposed females. Whatever the exposure treatment, virgin females were more likely to attack males than already mated females.

Johnson, 2005
Squinting bush brown butterfly

Bicyclus anynana
Biased learning Visual. Females exposed on the morning of their emergence for 3 h to a single virgin male with zero, 2 or 4 forewing dorsal spots. Naive females were not exposed and kept isolated until mate choice trial. Some males used for exposure had their sex pheromone producing organs blocked. 2-day-old female preference measured in a two-choice assay, between a zero and a 2-spot male, or between a 2 and a 4-spot male.
Females switch their innate preference from 2-spot males (on the forewing) to 4-spot males if exposed to them upon emergence. Females don't learn to prefer 0-spot males. When sex pheromones are blocked, females learn to avoid the 4-spot males. Westerman et al., 2012 Westerman andMonteiro, 2013 Squinting bush brown butterfly Bicyclus anynana

Learning
Olfactory. Females exposed on the morning of emergence to males with a wild-type sex pheromone blend, or a reduced sex pheromone blend (lacking one component and the two other highly reduced in quantity), or an enhanced blend (one component increased by perfuming). 2-day-old female preference measured in a two-choice assay with the wild-type and the reduced blend males, or with the wild-type and the enhanced blend males. Preference of 2-day-old naive female offspring of the females exposed to reduced blend, and of offspring of females exposed to wild-type blends measured in a two-choice assay with a reduced blend male and a wild-type blend male.
Females exposed to wild type blends find the reduced blend unattractive, but when exposed to the reduced blend, find this blend as attractive as the wild type blend. Females exposed to the enhanced blend learn to prefer it over the wild type blend. Offspring of females exposed to the reduced blend stop showing a preference for the wild type blend, compared to offspring of females exposed to wild type blends. This is the first study that shows inheritance of a learned response to a pheromone blend. Dion et al., 2017 African cotton leafworm moth Spodoptera littoralis Exposure Olfactory. Females exposed to a 0.1 female equivalent of the sex pheromone extract for 4 min, within 2 min from the beginning of scotophase or 3 h before the onset of the expected scotophase on the first 2 days after emergence. Control females were exposed with the solvent used for pheromone extraction. Female calling behavior measured in a wind tunnel during the 4 first days after emergence.
Exposed females increased the proportion of subsequent calling and called longer than non-exposed females. The effect was persistent for at least two additional days. This behavior could be advantageous under high population densities with high competition between females. Sadek et al., 2012 (Continued)  1-day-old female exposed housed with four 1-day-old females (calling, emitting sex pheromones) or housed alone. Female calling behavior recorded every night (for 5 nights) every 10 min for 130 min.
On the first night, exposed females signaled at a higher rate than females kept alone. The following nights, exposed females gradually signaled less than isolated females.

Harari et al., 2011
Hide beetle Dermestes maculatus Mating experience Full phenotype. 1-to 4-weekold females paired with males until copulation. 20 min after first copulation, females were paired with the same male or a novel male.
Females presented with a new male mated faster and rejected males less than when presented with their previous mate. Archer and Elgar, 1999 Banded demoiselle damselflies Calopteryx splendens C. virgo

Learning
Full phenotype and visual.
Males from the two different species are recognized thanks to their different wing patches. Sexually experienced females were caught in the field, or they were housed with each male morph for 2 h. Control females were reared isolated in the lab. Individuals from both species came from sympatric or allopatric populations. Female mate choice tested in a two-choice assay with tethered C. virgo and C. splendens males, before and after the social experience.
Virgin C. splendens females did not discriminate against heterospecific males based on wing patches, but experience, including mating, made them learn to reject heterospecific males. This learning helps species recognition in sympatric populations, and promotes reproductive isolation. Svensson et al., 2010Svensson et al., 2014 Banded demoiselle damselfly Calopteryx splendens Learning Visual. C. virgo male wing patch is fully melanized while only partially melanized in C. splendens. Experienced females were caught in the field, or exposed to a locally caught C. splendens male for 1 h followed by exposure to a C. virgo male, or vice versa. Control females were reared in isolation. Female preferences for male wing patch size tested by presenting conspecific tethered C. splendens males with manipulated wing patches varying in size.
Field experienced C. splendens females from sympatric populations with C. virgo preferred small male wing patches, whereas females from the allopatric population preferred large patches. Females from the sympatric population exposed to a conspecific male developed preference for smaller wing patch sizes, whereas females from the allopatric population exposed in the same way didn't develop such a preference. Control females didn't have a preference. Co-occurrence with a closely related species caused C. splendens females to prefer male traits that are more species specific.

Verzijden and Svensson, 2016
Treehopper Enchenopa binotata species complex Social plasticity Acoustic. 2-to 3-week-old virgin females (before their sexual receptivity) exposed for ∼2.5 weeks (50 min per day) to the preferred (the population peak preference) male call frequency, or to lower or higher frequencies (non-preferred, overlapping with heterospecific male calls), or to a mixture of call frequencies, or to silence. 4-to 5-week-old female response signal measured when tested with various male call frequencies.
The female preference curve was built to measure her selectivity and determine changes in peak preference (Figure 1).
Females preferred the mean call frequency of the males from the same population (this mean is the female peak preference). Females exposed to the preferred call frequency (their population peak preference) and females exposed to a mix of preferred and non-preferred call frequencies, increased their subsequent selectivity (lower responsiveness and tolerance and higher strength of preference) for the preferred call. Females exposed to silence, or non-preferred call frequency, didn't change their selectivity.

Fowler-Finn and Rodríguez, 2012a
Fowler-Finn and Rodríguez, 2012b Females reared in populations with controlled genetic background (full sib families), or in populations with different male calling rates, or in populations with different densities. After reaching adulthood, females were kept isolated. Sexually receptive female (∼7-weeks-old) response signal measured when tested with various male call frequencies. The female preference curve was built to measure her selectivity and determine changes in peak preferences (see Figure 1).
There was significant genetic variation in social background on peak preference and on female selectivity. Female preferences also varied with local density, with higher signal frequencies being preferred in denser environments. Rodríguez, 2013 Fowler-Finn et al., 2017 exposed to these males shortly during sexual maturation Westerman et al. (2012). Importantly, learning is context dependent, as female butterflies learn to avoid, rather than prefer, the novel wing patterns when the male sex pheromone is absent in the training male (Westerman and Monteiro, 2013). This suggests that olfactory communication may trump visual communication in assessing mates, at least in B. anynana butterflies. In addition, learning can be biased in that some sexual signals (supernumerous eyespots) can induce an increased preference while exposure to others (fewer eyespots) does not modify innate mate preferences (Westerman et al., 2012).

Olfactory Signals
In multiple studies, the learned sexual signal that triggers a behavioral change is an olfactory signal. Innate preferences appear to be generally present for olfactory signals in insects, and they can trigger a wide range of behaviors in receivers of many species from the same or the opposite gender. Odor learning can lead to either habituation or to sensitization, which led to opposite behavioral responses, but it is still unclear how the two processes work. Already ∼35 years ago, studies reported how male sweat bees avoided mating with previous mates, or with females genetically close to their first mate, by learning to recognize their particular odor (Barrows, 1975;Smith, 1983;Wcislo, 1987). Similar processes were recently reported in rove beetles and fruit flies (Schlechter-Helas et al., 2012;Tan et al., 2013). Such "habituation" to female odor also occurs in male moths, which reduce their level of response to female sex pheromones shortly after exposure to components of the blend (e.g., Robledo et al., 2018;Suckling et al., 2018). On the contrary, increased behavioral sensitivity to sex pheromones, termed "sensitization, " was also described in Spodoptera littoralis moths whereby sexually mature adult males increase their responsiveness and attraction to the odor source 24 h after exposure (Anderson et al., 2003(Anderson et al., , 2007Silvegren et al., 2005). Male S. littoralis also learned to prefer a mating site based on a learned plant odor, and reproduced preferentially with females found on the same plant where they grew up as larvae, or on plants where they previously mated Thöming et al., 2013;Proffit et al., 2015). Female B. anynana butterflies not only become sensitized to wild-type male sex pheromone composition, but can also learn to prefer unattractive blends if exposed to these blends during sexual maturation, right after adult emergence (Dion et al., 2017; Figure 2). Female moths also perceive the sex pheromone of other females, which induce them to emit their own sex pheromone earlier and at higher amount than inexperienced individuals (Stelinski et al., 2006;Sadek et al., 2012). Finally, the presence of antiaphrodisiacs, transferred by males onto the female cuticle, and reproductive tract led other males to learn to avoid mated Drosophila females (Ejima et al., 2005(Ejima et al., , 2007.

Acoustic Signals
Multiple studies have shown a change in sexual behavior upon exposure to acoustic signals, which are often used in species recognition and mate quality assessment in insects (Hedwig, 2016). For example, the rate and number of male calls that female crickets hear as juveniles or during mating significantly affects their preference and their response speed to future mate calls (Wagner et al., 2001;Rebar et al., 2011;Kasumovic et al., 2012). Contrary to individuals exposed to a mixture of call frequencies, females reared in silence respond faster toward a model song mimicking the populations' average calling rate Zuk, 2008, 2009;Bailey and Macleod, 2014;Swanger and Zuk, 2015). Males reared in silence intercept more females attracted to other males' calls and increase their own call rates (Bailey et al., 2010). Changes in a females' response to acoustic experience are variable and population-specific (Bailey and Zuk, 2012). The acoustic environment also impacts female treehopper's preference selectivity ( Figure 1C) for male signal frequency and speed (Fowler-Finn and Rodríguez, 2012a,b;Rebar and Rodriguez, 2016;Fowler-Finn et al., 2017). In addition, naïve female Drosophila initially show no preference to the courtship songs (wing vibrations) of conspecific or heterospecific males, but a pre-exposure to conspecific songs makes them prefer this song type (Li et al., 2018). In some parasitoid wasp species, males identify host pupae parasitized by a conspecific using acoustic and vibratory signals, learn their location, and visit them regularly, as a strategy to attain prospective emerging female mates (Danci et al., 2013(Danci et al., , 2014. This is one of the few cases where the adaptive value of learning is highlighted.  (Yeh and Servedio, 2015) and (Varela et al., 2018).

MOLECULAR MECHANISMS OF LEARNING
As detailed above, insects can change their sexual preferences and signaling upon social experiences and exposure to a variety of visual, odor, gustatory, or auditory signals, indicating that sexual behaviors are not fixed but plastic. The underlying molecular mechanisms that control this plasticity, however, are still largely unclear. Below we review a few mechanisms mediating such neural system plasticity. A social learning experience, such as courtship conditioning, where males experience female rejection in response to courtship, can lead to long-term changes in the behavior of males. This process of long-term memory consolidation in male Drosophila appears to depend on a peak of the ecdysteroid hormone, 20E, that appears immediately after the conditioning (Ishimoto et al., 2009).
Insects can learn to prefer (or avoid) a novel visual signal in a mate via early exposure to that signal but mechanisms of plasticity for preference development have only been explored in a non-sexual context. For instance, mRNA of three opsin genes in worker casts of the ant Camponotus rufipes increased upon exposure of these ants to daylight, as did volume of the three subneuropils of the optic lobe (including lamina, medulla, and lobula) (Yilmaz et al., 2016). A specific increase in UV and green opsin mRNA was also observed in the moth Helicoverpa armigera in response to 6 h exposure to UV light (Yan et al., 2014). These examples suggest that exposure of insects to particular visual signals displayed by the opposite sex could lead to changes in specific opsin expression levels as well as structural changes in the optic lobe, increasing sensitivity to those signals, and perhaps leading to later changes in sexual behaviors and preferences. This remains however to be investigated. Changes in protein expression levels and in cell size, cell number and cell connectivity of higher brain compartments in response to details of color patterns or courtship steps, rather than mere exposure to light of different colors, are also likely taking place but mediating mechanisms are still not known.
Mechanisms of pheromone odor sensitization have been explored to some extent in Spodoptera moths. In these experiments males are being briefly pre-exposed to a scent plume containing one or more components of the pheromone blend that increased their sensitization to the odor relative to naïve males (Anderson et al., 2003). The mechanisms that mediate this sensitization involve increases of the specific olfactory receptor expression and odor binding proteins in the antennae a few hours after the exposure (Wan et al., 2015), increased firing responses of the antennae (López et al., 2017) FIGURE 2 | Illustrations of learning sexual traits or preferences in insects and spiders. Learning can produce an acquisition or a shift of preference, or a change in the insect selectivity for the sexual signal. These processes can be due to mate choice or sexual signaling copying or to sensitization, which is an increase of the individual attraction to the stimuli upon exposure. All examples illustrate changes in the mate preference function, except for the courtship behavior copying, which illustrates a change in sexual signaling. and of the odor receptor neurons (Guerrieri et al., 2012), as well as changes in the size of the neural compartment processing the pheromone components in the olfactory lobes (Guerrieri et al., 2012). These physiological and structural changes have been hypothesized to lead to long-term memory of the early odor experience and stable changes in behavior (Anderson et al., 2007;Guerrieri et al., 2012).
Males learn to recognize mated or heterospecific Drosophila females thanks to the presence of cuticular hydrocarbons and antiaphrodisiacs transferred by the previous male onto the female's cuticle and reproductive tract (Ejima et al., 2005;Billeter et al., 2009). Recent work has identified neuronal differences across Drosophila species that are responsible for species-specific mate preferences regarding a female cuticular pheromone sensed by the legs of males (Seeholzer et al., 2018). It is possible that this conserved neuronal circuit, which is activated differently across species, is plastic and will be later implicated in learning of novel cuticular pheromones within a species, but this remains to be tested.
Drosophila males produce courtship songs by vibrating their wings. The song frequencies are perceived by the tip of the antennae, which detects air particle oscillations, and are processed by the Johnston's organ, at the base of the antennae (Ishikawa and Kamikouchi, 2016). The mechanisms that mediate the female's development of a song preference are still largely unknown but they involve signaling via the main inhibitory neurotransmitter, gamma-Aminobutyric acid (GABA), acting on the GABAA receptor Rdl in specific neurons (Li et al., 2018).
Currently there is limited evidence that learned mate preferences can be transmitted to the next generation in an insect and even more limited understanding regarding mechanisms. Daughters of B. anynana females exposed to novel sex pheromone blends show naïve preferences similar to those of their exposed mothers and different from non-exposed naïve individuals (Dion et al., 2017), but the mechanisms mediating the inheritance of this learned preference are unknown. Prolonged (5 day) olfactory conditioning in Drosophila was also inherited across two generations, but this work did not test the role of these learned odors on sexual behaviors (Williams, 2016).

MATHEMATICAL MODELS SHOW THAT LEARNING HAS AN EVOLUTIONARY IMPACT
In this section, we first briefly introduce models that assess whether learning itself can evolve and be selected as an evolutionary stable strategy (ESS), i.e., a strategy that if adopted by a population in a given environment cannot be invaded by any alternative strategy that is initially rare. If learning sexual traits is an ESS, then this implies that learning sexual traits is adaptive. Whether learning sexual traits affects their evolution and impacts speciation has also mostly been addressed with theoretical models that we also reviewed in this section.
The first set of models reveal that learning can be selected as an ESS, which is a prerequisite for learning to affect the evolution of sexual signals, mate preferences, and reproductive isolation (reviewed in Galef and Laland, 2005;Vakirtzis, 2011;Verzijden et al., 2012;Dukas, 2013;Servedio and Dukas, 2013;Witte et al., 2015;Head et al., 2016;Kopp et al., 2018;Varela et al., 2018). Overall, for learning to evolve under selection there needs to be genetic variation for learning ability within a species (Mery and Kawecki, 2005). This appears to be the case. An example involves the "rover" and "sitter" alleles at the foraging locus of D. melanogaster that confer different learning abilities to fly larvae when foraging for food (Mery et al., 2007;Papaj and Snell-Rood, 2007;Mery, 2013).
The second set of models assess whether learning affects the evolution of sexual traits and impacts speciation. These models were originally designed for sexual interactions in vertebrates, but here we focused on those models that can be applied to insects (Supplementary Table 1). Most of these models were built on the premise that sexual signals and mate preferences have a genetic basis (e.g., Ritchie, 2000;Shaw, 2000;Noor et al., 2001) that can be modified and be overridden by learning (Supplementary Table 1). The models usually focus on one of the following three underlying mechanisms of learning: "learning by sexual imprinting, " "learning by copying, " and "learning from previous experience" (Table 3 and Supplementary Table 1).
The first group of models focused on imprinting, which occurs when juveniles up to a certain age can learn a sexual preference by observing the phenotypes of surrounding adults (Immelmann, 1975;Head et al., 2016). In insects, the terms "early experience, " or "early exposure" to other individuals of the same generation are used instead of "imprinting" ( Table 3). This is primarily because there is still no data on whether or not insects have a fixed period in development or early adulthood when they can learn a preference from a social experience, as in the case of birds, where the term imprinting was first used (Lorenz, 1935). In insects, sexual imprinting of mate preferences can occur between genetically unrelated individuals of the previous or of the same generation, and is termed oblique or horizontal imprinting, respectively ( Table 3). Oblique or horizontal imprinting have limited effect on the evolution of sexual preferences and of reproductive isolation, except when spatial structure is taken into account. Spatial structure in models assumes that social learning is only possible between individuals that can perceive each other (Yeh and Servedio, 2015), i.e., that are close in space. Differentiation in social interactions between populations due to spatial isolation is expected to accelerate divergence of sexual preferences, signals, and of reproductive isolation between populations, through coupling of the divergent sexual signals and mate preferences across space (Bailey and Moore, 2012). If populations are exchanging migrants, oblique imprinting of mate preferences cannot produce sympatric speciation (Verzijden et al., 2007), and the populations cannot maintain genetic differentiation in their sexual traits (Yeh and Servedio, 2015). Interestingly, aversive learning of mate preference through oblique imprinting, when individuals learn to avoid a phenotype, was shown to accelerate reproductive isolation and to produce adaptive radiations (Gilman and Kozak, 2015). Finally, imprinting may contribute to reproductive isolation in insects through selfimprinting (i.e., self-referent phenotype matching) that facilitates reinforcement between incipient divergent lineages (Servedio et al., 2009).
Imprinting can also affect the expression of sexual signals, and these learned signals can also contribute to reproductive isolation (Williams and Slater, 1990;Ellers and Slabbekoorn, 2003;Lachlan and Servedio, 2004;Olofsson and Servedio, 2008;Olofsson et al., 2011;Planqué et al., 2014). These models often incorporate a spatial structure, but it is unclear if they can be applicable to insects because they assume that males learn to produce their sexual signals by imitating adults surrounding them during their development, which has so far has been documented only in the wolf spider Schizocosa ocreata (Clark et al., 2012(Clark et al., , 2015. These models also assume that assortative mating takes place between females and males that have learned to prefer, or express, a similar sexual signal by experiencing it locally, while assortative mating based on similarly preferred and expressed sexual traits does not occur in insects, as far as we know. A second mechanism of learning sexual behaviors in insects is to copy another individual's mating decision. Insects that mate in groups, such as promiscuous or lekking species [e.g., some species of ants, bees, paper wasps, and butterflies (Litte, 1979;Wickman and Jansson, 1997;Velthuis et al., 2005;Izzo and Tibbetts, 2012;Prato and Soares, 2013)] can modify their mating preference or the production of their own sexual signals by observing the success of other individuals mating. Most work has focused on "mate choice copying" ("MCC" hereafter), which is usually modeled as "positive" such that individuals (usually females) learn to prefer the phenotype of males that they have observed mating earlier (Servedio and Kirkpatrick, 1996;Santos et al., 2017). The copying behavior itself can spread in a population both through direct (Dugatkin and Höglund, 1995;Stöhr, 1998) or indirect (Servedio and Kirkpatrick, 1996;Santos et al., 2017) selection. MCC has direct selective benefits, if it reduces the sampling costs and/or the error rate of mate choice (Dugatkin and Höglund, 1995;Stöhr, 1998;Agrawal, 2001), but also has indirect selective benefits (Kirkpatrick and Dugatkin, 1994;Servedio and Kirkpatrick, 1996;Santos et al., 2017). These indirect benefits arise because females that copy others are more likely to mate with males that are attractive to other females, spreading in the process genes for attractive sons and genes for daughters with the ability to copy others. MCC can also both increase the variance in male sexual signals (Wade and Pruett-Jones, 1990), and erode genetic variance by eliminating novel or rare male signals, even if these males are fitter than the common males in the population (Kirkpatrick and Dugatkin, 1994). However, when biases in learning are present in mate choice copying, such that females are more strongly affected by experiences involving unusual stimuli (e.g., rare male phenotypes) than those involving standard stimuli (e.g., common male phenotypes), MCC can cause novel male signals to sweep through the population even if there is no inherent preference for the novel trait (Agrawal, 2001). Invasion of a novel sexual signal can also occur when "negative" MCC (aversive learning) is modeled, where females learn to avoid males avoided by other females (Santos et al., 2014). While Kirkpatrick and Dugatkin (1994) suggest that MCC may promote or accelerate population divergence, the role of MCC in reproductive isolation and speciation remains an open question (Varela et al., 2018).
A third learning mechanism occurs through previous (socalled "private" or "personal") experiences of either courtship or of actual mating events, during which mate preferences and the expression of sexual signals can be learned (Servedio and Dukas, 2013;Morier-Genoud and Kawecki, 2015). Females learning to prefer local or familiar (previously encountered) males increases the rate of divergence between spatially structured populations (Bailey and Moore, 2012), and also in case of a secondary contact (Servedio and Dukas, 2013). In contrast, when males learn to prefer local or familiar females, population divergence can be reduced because competition for accessing these females increases. Heterospecific males, which seldom meet heterospecific females locally, don't learn to prefer them as much as conspecific males, and keep courting and mating with both types of females (Servedio and Dukas, 2013). When males learn to improve the expression of their sexual signals through repeated courtship events, this accelerates the evolution of the sexual signal, even when the signal is costly, and it favors the emergence and spread of a novel male sexual signal, even in the presence of gene flow (Morier-Genoud and Kawecki, 2015).
Finally, some models compared the fitness advantage, or the likelihood of various mechanisms of mate preference learning to spread as evolutionary stable strategies (ESS). Learned mate preferences from previous encounters with potential mates increase fitness compared to other mate selection mechanisms including threshold-based mate preference (Dubois et al., 2012). Furthermore, the advantage provided by learning increases when variance in the quality among males increases locally, and across space or time (Collins et al., 2006). Depending on associated costs, learning mate preferences either through MCC or through previous personal experience can both coexist as an ESS in mixed populations of females displaying either one or the other learning mechanisms (Dubois et al., 2012).

CONCLUSIONS AND PERSPECTIVES
Experimental work on the role of learning in sexual interactions in insects and spiders is a burgeoning field revealing that the traditional view that insects are small robots with mostly innate, genetically fixed sexual behaviors, is now obsolete. Learning in sexual interactions is the rule rather than the exception in every organism tested so far. Innate, genetically fixed sexual preferences and signals are present only in some species, and are more commonly observed in specific modes of communication such as olfactory signals, whereas learning a sexual preference or the expression of a sexual signal is widespread. Learning also affects the expression of innate sexual traits. Learning is usually assumed to be positive, but it can also be negative (i.e., aversive), as well as biased and contextdependent.
Our review revealed the diversity of terminology used by authors to describe experiments involving learning [ Table 1, 2, column "Type of learning (as per the authors)"], which can be confusing and prevent the identification of the underlying mechanisms. Hence, we would like to encourage researchers to provide explicit details of their methods as described in Tables 1-3 (e.g., the developmental stage at which the learning happens, the sex of the demonstrator and of the insects that learns). This information will specify the processes of learning used by insects, e.g., sensitization, simple exposure, or conditioning, which will help identify the underlying neurophysiological mechanisms involved.
Finally, most scientists assume that learning sexual traits has evolved under selection, as appears to be the case in vertebrates (Morand-Ferron, 2017) and regarding other behaviors in insects (Nieberding et al., 2018), but there is little to no evidence that learning sexual traits affects insect fitness, particularly in the wild. This is perhaps because one of the first definition of learning included adaptation [learning is an "adaptive change in individual behavior as the result of experience;" Thorpe (1963)]. We encourage field work to complement laboratory experiments with ecologically-relevant setups to quantify the adaptive value of learned sexual interactions across insects. Showing the adaptive value of such learning would explain its prevailing presence in such miniature brained, short lived, organisms.

AUTHOR CONTRIBUTIONS
ED, AM, and CN conceived the scope of the review, collected information from the literature, and contributed to the writing and editing of the manuscript. All authors contributed to revisions.