Recent Developments in the Remote Radio Control of Insect Flight

The continuing miniaturization of digital circuits and the development of low power radio systems coupled with continuing studies into the neurophysiology and dynamics of insect flight are enabling a new class of implantable interfaces capable of controlling insects in free flight for extended periods. We provide context for these developments, review the state-of-the-art and discuss future directions in this field.

Building very small flyers which can fly well in real environments is a difficult task, however, hampered not just by knowledge of insect flight, but current technological limitations. A housefly is about 8 mm long and weighs 1 g; its wings beat approximately 200 times a second; during each stroke, tiny muscles make subtle adjustments to the trajectory of the wings to generate the forces required to move it where it wants to go Dickinson, 1994, 1996;Dickinson and Tu, 1997;Dickinson et al., 1999Dickinson et al., , 2000. As it does this, it integrates data from a variety of sensors on its body which provide airflow information, gyroscopic information, and visual flow information; it computes and then compensates for the horizon, for incoming obstacles, and for constantly changing wind gusts and does this so efficiently, that it can fly for hours a day and could fly a mile or two in single stretch if really had to. By comparison, the Delfly Micro (de Croon et al., 2009), arguably one of the smallest untethered, instrumented air vehicles, weighs 3 g and has a 10 cm wingspan, can fly for 3 min on a battery which constitutes 1/3 of its weight and requires

IntroductIon
Insects exhibit some amazing modes of locomotion. They float, swim, crawl, jump, run (even bipedally), and fly. They do this with such efficiency, sophistication, and elegance of function across a vast size range (the smallest insect is 1/5 of a millimeter across, the largest insects can weigh 100 g and be over 20 cm long) that they have mastered locomotion on almost every terrain on Earth.
The adaptations which enable insects to do these things have inspired human art, science, and engineering for centuries (e.g., Tipton, 1976). On the ground, the science of insect locomotion has informed several generations of legged robots (e.g., Full and Koditschek, 1999;Delcomyn, 2004;Lehmann, 2004;Riztman et al., 2004), from the crawling machines of the 1980s (like MIT's Ghengis; Brooks, 1991) to the more modern Rhex capable of successfully traversing complex terrain (Altendorfer et al., 2001). In the air, a great deal of research spanning decades is beginning to tease out the complex interplay between aerodynamics, mechanical construction, and neurophysiology that enables something as small as a fruitfly to fly so incredibly well. Enabled by advances in microfabrication techniques (Tanaka and Wood, 2010) and considerable progress in our understanding of insect flight (Pringle, 1957;Pringle, 1959, 1960;Machin et al., 1962;Ikeda Frontiers in Neuroscience www.frontiersin.org December 2010 | Volume 4 | Article 199 | 2 dual channel telemetry backpack to measure and compare EMG's from a pair of flight muscles of a male hawkmoth, Agrius convolvuli, during pheromone-triggered zigzag flight (Kuwana et al., 1999;Ando et al., 2002;Ando and Kanzaki, 2004;Wang et al., 2008). To our best knowledge, three different groups have recently developed wireless systems that can both transmit and receive data from freeflying insects flight control (Figures 1-3). Flight control requiring multiple-channel stimulation, during complex, long-duration controlled flight requires on-board digital processing, memory, and programmability in addition to efficient radio systems. Each group developed systems with different trade-off choices in terms of functionality, weight, and complexity. Sato et al. describe an 8-channel system built around the Texas Instruments CC2431 microcontroller with built-in transceiver; careful programming of the microcontroller allows for half hour of flight time and approximately 24 h of battery life in sleep mode (Figure 1, Sato et al., 2009a,b;Maharbiz and Sato, 2010). The use of surface mount ceramic antennas results in a very small package in terms of size, mass, and inertial effect on the flying insect ( Figure 1B). Bozkurt et al. (2009b) developed a custom-built, 2 channel AM receiver which used pulse-position modulation via a super-regenerative architecture which was fed into a PIC12F615 microcontroller (Figure 2). Daly et al. (2009Daly et al. ( , 2010 developed a custom silicon system-on-chip receiver operating at 3-5 GHz on the 802.15.4a wireless standard which interfaced with an on-board Texas Instruments MP430 microcontroller; the receiver was remarkable for its extremely lowpower operation (2.5 mW, 1.4 nJ/bit) for a data rate of 16 Mb/s (Figure 3). Driven primarily by technological developments in ultra-low power distributed sensor networks, low power microcontrollers equipped with internal radios are now very accessible.

StImulatIon protocolS
Flight control of insects ideally requires the triggering of flight initiation and cessation as well as the free-flight adjustment of orientation with 3 degrees of freedom (Dudley, 2000;Taylor, 2001). It is important to note that all published attempts at free flight control rely on the insect to "fly itself " while periodically introducing extraneous input to bias the free flight trajectory. A sufficiently sophisticated system is, in effect, wrapping a synthetic control loop around the existing biological one; the idea of interfering with a biological control loop using an extraneous, synthetic loop a human operator to perform all flight functions. To the authors' knowledge, the smallest, non-autonomous flying machine is the microrobotic fly built at the Harvard Microrobotics Laboratory, weighing in at 60 mg (Wood, 2008). While these systems are rapidly evolving, they are currently hampered by the energy and power density of existing fuel sources and the difficulty in replicating the flight dynamics and mechanical efficiencies of very small flyers. Insects have flight performance (as measured by distance and speed vs. payload and maneuverability) as-yet unmatched by man-made craft of similar size. Very recently, several groups have attempted to circumvent such problems by merging synthetic control and communication systems into living insects with the aim to control free flight.

technology WIreleSS telemetry and control
The development of wireless telemetry systems for small, free-flying, and walking insects began in the early 1990s with applications ranging from neuromuscular recording (Kutsch et al., 1993(Kutsch et al., , 2003Kuwana et al., 1995Kuwana et al., , 1999Fischer et al., 1996;Holzer and Shimoyama, 1997;Fischer and Ebert, 1999;Fischer and Kutsch, 1999;Ando et al., 2002;Kutsch, 2002;Ando and Kanzaki, 2004;Colot et al., 2004;Cooke et al., 2004;Takeuchi and Shimoyama, 2004;Mohseni et al., 2005;Lemmerhirt et al., 2006;Wang et al., 2008) to using radio transmitters to study the long-range movements of insects (Hedin and Ranius, 2002;Sword et al., 2005;Holland, 2006;Wikelski et al., 2006Wikelski et al., , 2010Pasquet et al., 2008). Table 1 shows the wireless systems used for neuromuscular recording and stimulation of freeflying insects. All of the early devices employed custom-made radios hand-assembled from surface mount electronics components and lacked digital processing, on-board memory or programmability. Detection of emitted signals was usually carried out using complimentary radio receivers or spectrum analyzers (and the relevant biological information was de-convolved from the analog radio signals). Kutsch et al. (1993) developed a 0.42 g telemetry backpack (including battery) for a locust (Schistocerca gregaria, 3.0-3.5 g, payload capacity 0.5 g). The backpack had a single channel transmitter to wirelessly acquire electromyograms (EMG) of a single flight muscle of interest. Their modified backpack had a dual channel transmitter and it allowed the researchers to tease out the function of the locust's proprioceptors (Fischer and Ebert, 1999), a feat difficult or impossible to do with tethered insects. Ando et al. developed a 0.23 g

Telemetry
The science and technology of measuring, transmitting, and storing signals of interest remotely. An instrument to measure and transduce the phenomenon of interest into electrical signals is attached on a target and the electrical signals are modulated and transmitted to the users' receiver, which demodulates the electrical signals. has a long history. In insects, the motif has been used repeatedly in studies of motor control and biomechanics (Nishikawa et al., 2007). To date, wireless flight control of insects has relied on either neuromuscular or neuronal stimulation. In either case, the chosen interface and complimentary stimulation protocol (i.e., electrode geometry, electrode implantation method, stimulus conditions) must generate reproducible, quantifiable alterations to insect flight in a way that is robust to the harsh conditions before, during and after free flight. Free-flying insects routinely impact objects (shocks and hard impact are observed not just in flight or during accidents, but very often while landing); the vibrations of the center of mass can be substantial and at frequencies (50-200 Hz) which can resonantly couple to extraneous mechanical components (i.e., 3 cm dipole antennas); and the legs or wings themselves can interfere with operation during normal flight. All of these conditions invariably lead to mechanical drift of the implanted electrodes over the lifetime of the insects. Successful, robust stimulation schemes in free flight have thus focused on combinations of the following three motifs: (a) the direct stimulation of a large, easily accessible muscle in the insect (Bozkurt et al., 2008a(Bozkurt et al., , 2009aSato et al., 2008aSato et al., ,b, 2009aMaharbiz and Sato, 2010) steer in free flight (Movie S1 in Supplementary Material, Figure 4A, Sato et al., 2008bSato et al., , 2009aMaharbiz and Sato, 2010). Ten insects initiated flight in response to stimulation, with the median number of stimulation waveforms required to initiate flight being 19 (range 1-59). One stimulation waveform was a pair of biphasic square pulses (1 ms per each pulse, 4 ms pitch). The median response time from the first stimulation to flight initiation being 0.5 s (range 0.2-1.4 s).
Median flight duration in response to stimulation was 46 s (range 33-2292 s). Stimulation voltage between 2 and 4 V did not affect the number of stimuli required to initiate flight, response time from stimulation to flight, or flight duration in Mecynorhina torquata (Mann-Whitney U-tests, P = 0.13, 0.46, 0.35, respectively). Data on stimulated flight bouts in individual beetles are summarized in Sato et al. (2009b). Once a ganglion (Sato et al., 2008a(Sato et al., ,b, 2009aBozkurt et al., 2009b), (c) the targeted stimulation of nerves in a nerve cord (Tsang et al., 2008(Tsang et al., , 2010aDaly et al., 2009Daly et al., , 2010.

FlIght InItIatIon and ceSSatIon
In adult Mecynorhina ugandensis beetles, the abrupt darkening of the environment during untethered free flight led to the almost immediate cessation of flight. This led us to hypothesize that light levels and corresponding changes in neural activity at the optic lobes might strongly modulate flight initiation and cessation. In fact, potential pulses applied between two electrodes implanted near the base of the left and right optic lobes could elicit flight initiation and cessation with very high success rates. Implantation into the optic lobe yielded a much higher success rate and did not affect the beetle's ability to

Optic lobe
The part of the protocerebrum (one of the three major regions of insect brain) that extends to the retina of the visual receptors including compound eye and ocelli). Neural signals induced at the visual receptors descend to the optic lobe. The optic lobe consists of three large neuropils: lamina, medulla, and lobuli. given insects, flight bout duration was correlated with neither beetle mass nor stimulus amplitude. Furthermore, the beetle adopted a normal flight posture and continued flying in the air after the stimulus was turned off, indicating that the tonic neural signals required for flight maintenance continued after stimulus. A relatively long duration pulse applied between optic lobes, which effectively clamps the voltage between the lobes, stopped flight for Mecynorhina torquata. Ten insects were tested in flight was initiated by our stimulation, the flight tended to persist without additional stimulation whether the beetle was either in the tethered or in free flight condition. During normal flight, the beetle nervous system produces a pulse train with approximately 50 ms period to the basalar muscles (Josephson et al., 2000a,b). Alternating positive and negative potential pulses at 100 Hz applied between left and right optic lobes initiated wing oscillations while a single pulse ceased wing oscillations; (top) audio recording of tethered beetle, (bottom) applied potential to the one side optic lobe regarding the other side optic lobe. The sharp rise of audio amplitude at the beginning of oscillation is attributed to friction between elytra and wings when the wings came out from the underneath of elytra. The whole audio amplitudes were normalized by mean absolute value calculated for the middle period of the flight time (2.5-3.7 s). (B) Elevation control of a free-flying beetle: temporal height change of a flying beetle (10 flight paths). Alternating positive and negative potential pulse trains at 100 Hz and 2.0 V amplitude to the brain caused the beetle to fly downward. The median height change was 60 cm (the range was 33-129 cm). (C,D) Turn control of free-flying Mecynorhina torquata beetle. Pulse trains at 100 Hz and 1.3 V positive potential to the left or right basalar muscle triggered turns. Ten flight paths elicited by (A) right or (B) left basalar muscle stimulus for 0.5 s. Each flight path is obtained after the three-dimensional digitized flight path is projected on the XY plane (see Sato et al. 2009b). Different colored and shaped plots show individual beetle flight paths. See Movies S1-S4 in Supplementary Material for the initiation and cessation in free flight (Movie S1), elevation (Movie S2: tethered, Movie S3: free flight) and turn controls (Movie S4). Reproduced from Sato et al. (2009b).

Basalar muscle
One of the flight muscles in insects. In beetles (Coleoptera), the basalar muscle is directly connected to the wing base via the apodema cap. The basalar muscle is a well-developed fibrillar muscle composed of approximately 90 fibers in C. texana (Josephson et al., 2000b). The contraction of the basalar muscle produces the downstroke of wing oscillation. See Figure 1 for the location and structure of the basalar muscle of Coleoptera. strong motions of the head. In fact, the contraction of flight muscles is usually preceded by the rotation of the head toward the aimed direction (Berthoz et al., 1992). Exploiting this, Bozkurt et al. (2009b) stimulated neck muscles to induce turning in flying moths. An electrical stimulus delivered to the neck muscles via thin wire electrodes implanted in the Manduca sexta elicited yawing in balloon-assisted flight (Figure 5). In contrast to attempts at direct stimulation of the wing muscles, neck-muscle stimulation avoids damage to the complex linkages and muscles of the wing. For insects whose small size or wing muscle complexity makes direct wing muscle stimulation prohibitive (i.e., bees, flies), this method has decided advantages. Tsang et al. (2008Tsang et al. ( , 2010a developed a microfabricated polyimide multi-site flexible splitring electrode (FSE) which could be implanted around the insect's ventral nerve cord just below the fourth abdominal segment during stage-16 of pupation (2 days prior to emergence). This approach was informed by the fact that changes in an insect's center of gravity can be used to adjust flight orientation and trajectory (Ellington, 1984). In moths, stimulation of the ventral cord with tungsten wires elicited abdominal motions, "presumably by activating motoneurons or interganglionic interneurons" (Tsang et al., 2008(Tsang et al., , 2010a. Each FSE contained six independently addressable electrodes; potential pulse trains were applied between pairs of electrodes on the FSE (for a total of 15 possible stimulation pairs). The application of 1-500 ms, 1-5 V potential pulses with frequencies varying between 50 and 333 Hz, elicited directional contraction of the abdomen depending on the electrode pair chosen. Interestingly, the direction of contraction for a given electrode pair not only varied from animal to animal but between the pupal and adult stages of the same animal, implying not only "movement of the FSE but probably also … developmental differences in the location and identity of axons in the nerve cord and changes in the mechanical articulation of the abdomen." By adjusting voltage levels and frequency, the abdominal response could be graded, an important consideration for future studies of free flight control. Abdominal contractions in loosely tethered moths elicited by the FSE were shown to correspond to changes in flight path (Figure 6, Tsang et al., 2010a).
Asymmetric stimulation of the muscles that actuate insect's wings can be used to generate turns. In beetles, for instance, turns could be elicited by stimulus of the left and right basalar muscles with positive potential pulse trains. In C. texana, the basalar muscles normally contract and extend at tethered condition and each test was repeated for 10 times, i.e., 100 tests in total. Data on cessation of flight in individual insects are summarized in Sato et al. (2009b). All 10 insects tested were forced to stop flying by amplitude of 6.0 V or less. The majority (77%) stopped with a 2-3 V amplitude. The median amplitude was 3.0 V (range 2-6 V). The majority (87%) showed quite short response time <100 ms. In Manduca sexta, Bozkurt et al. (2009b) showed that stimulation of the antennal lobes with 20 Hz, 3.5 V peak-to-peak pulses elicited flight while stimulation of the same site with 50 Hz, 3.5 V peak-to-peak pulses ceased flight.

throttlIng the oScIllator
During flight, the frequency and stroke amplitude of wing oscillation could be manipulated with the neural stimulator in beetles (Sato et al., 2008a(Sato et al., , 2009bMaharbiz and Sato, 2010). For C. texana, it was observed that progressively shortening the time between positive and negative potential pulses delivered to the area of the brain between the optic lobes led to the "throttling" of flight where the beetle's normal 76 Hz wing oscillation was strongly modulated by the 0.1-10 Hz applied stimulus (Sato et al., 2008a(Sato et al., , 2009b. A repeating program of 3 s, 10 Hz, 3.0 V pulse trains followed by a 3 s pause (no stimulus) resulted in alternating periods of higher and lower pitch flight (Sato et al., 2008a(Sato et al., , 2009a. In a similar fashion, Mecynorhina torquata, brain stimulus at 100 Hz led to depression of flight ( Figure 4B, tethered flight: Movie S2 in Supplementary Material, free flight: Movie S3 in Supplementary Material). Set on a custom pitching gimbal, Mecynorhina torquata could be repeatedly made to lower angle to horizon when stimulated. The change in the length of envelope of the blurry region around the wing suggests that the wing stroke amplitude was clearly reduced (see the tethered flight: Movie S2 in Supplementary Material). Ten of 11 tested beetles showed the tendency (Sato et al., 2008b(Sato et al., , 2009aMaharbiz and Sato, 2010).

controlled turnIng
One of the classical methods for studying flight control in tethered animals is through the use of changing visual cues within the insect's field of view. Employing arrays of light emitting diodes (LED's) or digital projection on a screen, visual features such as scrolling stripes, moving shapes, and changing horizons elicit very strong maneuvering responses from many insects (e.g., Bothoz et al., 1992, Tu and Dickinson, 1996, Sato et al., 2008a. Given that most insect compound eyes cannot move to track targets, visual cues which induce locomotion responses often also elicit Gimbal A structure consisting of a frame which can rotate about a pivot point, constraining the rotation of the frame and anything attached to it to a single axis. Nesting more than one pivoted frame allows for rotation about multiple axes. For the custom gimbal discussed here, the frame was a plastic ring attached to a support ring using elastomeric material (which formed a pivot point). The inner ring had a magnetic attachment point where a flying insect was localized; when the insect was stimulated the amount of rotation about a single axis could be measured (e.g., pitch angle).

Frontiers in Neuroscience www.frontiersin.org
December 2010 | Volume 4 | Article 199 | 8 directions. Undoubtedly, one of the outstanding issues is determining to what extent sophisticated, synthetic control can bias complex flight in natural environments; this is an old topic in the insect neurophysiology community but one which may deserve re-evaluation given rapid advances in interface technology and the extreme miniaturization of computation. How much control effort (and, thus, energy) must be expended to direct an insect along a given trajectory in environments which present extraneous stimuli? Are there inherent limits? To what extent can these neural and neuromuscular responses and their resultant trajectory changes be reliably graded? Put differently, can a reliable servo description for these interfaces be encoded as a pre-requisite for attempting control via local (on-board) algorithms? Beyond the issue of control, insects which undergo complete metamorphosis may present a unique system with which to study syntheticorganic interfaces; an idea recently posited in Paul et al. (2006). Several groups have begun to explore the advantages, in mechanical, electrical or surgical contexts, of interfaces implanted in insects during pupation, i.e., prior to emergence as adults (Paul et al., 2006;Bozkurt et al., 2007Bozkurt et al., , 2008aBozkurt et al., ,b, 2009aTsang et al., 2008Tsang et al., , 2010aSato et al., 2008b;Chung and Erickson, 2009). Given the extensive re-working of the insect physiology 76 Hz when they are stimulated by approximately 8 Hz neural impulses from the beetle nervous system (Josephson et al., 2000a,b). It has been reported that the flight muscles in Cotinis produce maximum power when they are stimulated directly by electrical pulses at 100 Hz (Josephson et al., 2000b). During flight of tethered C. texana, a turn could be elicited by applying 2.0 V, 100 Hz positive potential pulse trains to the basalar muscle opposite to the intended turn direction (Sato et al., 2008a(Sato et al., ,b, 2009aMaharbiz and Sato, 2010). A right turn, for example, was triggered by stimulating the left basalar muscle. In free-flying Mecynorhina torquata, turning was elicited when either of the left or right basalar muscles was stimulated in the same manner as C. texana but 1.3 V (Movie S4 in Supplementary Material, Figures 4C,D, Sato et al., 2009a,b;Maharbiz and Sato, 2010). The success rates for left and right turn were 78% (N = 42) and 66% (N = 68), respectively. One second of left-and right-stimulation of free-flying beetles resulted in a 1.7 and −9.0 median roll to the ground and a 20.0 and 32.4 median rotations parallel to the ground, respectively (see, Sato et al., 2009b for data on stimulus turns in free-flying Mecynorhina torquata).

ongoIng Work and applIcatIonS
From an engineering perspective, the latest push toward remotely controlled free-flying insects is providing impetus for several new research

Complete metamorphosis
One of the two major types of metamorphosis (the other is incomplete metamorphosis). Metamorphosis is the biological process that insects undergo in the development from larva (or nymph) to adult (or imago). In metamorphosis, the insect's body structure, habitat, and behavior often distinctly change. Insects which undergo complete metamorphosis pass through three different structural and habitudinal stages including larva, pupa, and adult. Insects which undergo incomplete metamorphosis progressively enlarge via ecdysis (molting or the shedding of the exoskeleton) but keep similar structure and habit from larva to adult. of insect communication, pollination and mating behavior and flight energetics, and for studying the foraging behavior of insect predators such as birds, as has been done with terrestrial robots (Michelsen et al., 1989). The technology may also enable new types of experiments relevant to neuroscience as it relates to insect flight. Remote stimulation and recording systems, coupled with flight arenas equipped with real-time motion-capture systems, can trigger motor responses decoupled from the insect's sensory inputs while tracking the resultant changes in flight behavior and recovery. This can be done while simultaneously recording neural or neuromuscular signals from the insect. This is an area of interest in our lab, specifically as it informs or improves the ability to elicit controlled reactions from flying insects. Moreover, the ability to take real-time motion data allows for the timing of control signals referenced to a specific state of the insect in flight (e.g., stimulation is applied only at specific orientations, velocities, rotations, etc.) which may help tease out the control circuits at work in the insect. In engineering, during pupation, it is tempting to hypothesize that interfaces inserted during this period could somehow co-opt the developmental processes for an engineering advantage; this has not yet been conclusively shown. The process does provide engineering advantages, however. For example, both the authors and others have found that insertion of foreign objects trans-cutaneously in the pupal stages often results in mechanically robust implants as the shell hardens around the structure post-emergence (Paul et al., 2006;Sato et al., 2008b). Complex surgeries, as for that required in Tsang et al. (2008Tsang et al. ( , 2010a, are easier in pupa than adults. Moreover, for insects with large interstitial areas (such as horned beetles), a significant mass of material can be introduced into the pupa which becomes incorporated in the insect (provided neither nerves, gut, nor muscle was severed in the insertion, Bozkurt et al., 2008b). The ability to control the flight of insects and receive information from on-board sensors would have many applications. In biology, the ability to control insect flight would be useful for studies electronically controllable insects could be useful models for insect-mimicking M/NAV's (Micro/ Nano Air Vehicles) (Wu et al., 2003;Schenato et al., 2004;Deng et al., 2006a,b;Wood, 2008). Furthermore, tetherless, electrically controllable insects themselves could be used as M/NAV's and serve as couriers to locations not easily accessible to humans or terrestrial robots.
In engineering, electronically controllable insects could be useful models for insect-mimicking M/NAV's (Wu et al., 2003;Schenato et al., 2004;Wood, 2008). Furthermore, tetherless, electrically controllable insects themselves could be used as M/NAV's and serve as couriers to locations not easily accessible to humans or terrestrial robots. Lastly, and perhaps most importantly, these systems provide a readily accessible platform with which to study the integration between man-made interfaces and multicellular organisms engaged in complex tasks. This endeavor will certainly not replace the pursuit of building synthetic flying robots (since humans often build better machines than nature does), but computation and communication technology scales faster rate than power supply energy density or mechanical actuation. As smaller and lower power microcontrollers and radios continue to appear on the market, researchers will be able to add an increasing amount of synthetic control into organic systems enabling new classes of programmable machines.

Micro/nano air vehicles
A class of small-sized remote-controlled unmanned aerial vehicles (UAV). In a restricted definition, micro air vehicles (MAV) and nano air vehicles (NAV) are less than 150 and 75 mm in maximum length, respectively. In many cases, the design, mechanics and aerodynamics of M/NAV's has been inspired by bird and insect flight (Wu et al., 2003;Deng et al., 2006a,b;Wood, 2008;de Croon et al., 2009).