Sensing and Biosensing in the World of Autonomous Machines and Intelligent Systems

In this paper we discuss how nanotech-based sensors and biosensors are providing the data for autonomous machines and intelligent systems, using two metaphors to exemplify the convergence between nanotechnology and artificial intelligence (AI). These are related to sensors to mimic the five human senses, and integration of data from varied sources and natures into an intelligent system to manage autonomous services, as in a train station.


INTRODUCTION
The rapid progress in autonomous systems with artificial intelligence (AI) has brought an expectation that machines and software systems will soon be able to perform intellectual tasks as efficiently as humans (perhaps even better), to the extent that in a near future, for the first time in history such systems may be generating knowledge, with no human intervention (Rodrigues et al., 2021). This tremendous achievement will only be realized if these manmade systems can acquire, process and make sense of a lot of combined data from the environment, in addition to mastering natural languages. The latter requirement appears particularly challenging since machine learning (ML) and other currently successful AI approaches are not yet sufficient to interpret text (Rodrigues et al., 2021). Another stringent requirement is in the capability of continuously acquiring information with sensors and biosensors, in many cases having to emulate human capabilities. As for shorter-term applications, medical diagnosis and any other type of diagnosis are among the topics that may benefit most from AI. This is due to a convergence with multiple technologies that are crucial for diagnosis, namely the nanotech-based methodologies which allow for ubiquitous sensing and biosensing to be integrated into diagnosis and surveillance systems (Rodrigues et al., 2016). Diagnosis is essentially a classification task, for which ML has been proven especially suited, in spite of its limitations in performing tasks that require interpretation (Buscaglia et al., 2021).
Living beings depend on sensing for their survival, growth, reproduction, and interaction. Humans, in particular, use their five senses (touch, sight, hearing, smell and taste) at all times to monitor their environment and interact with it, where most of the sensorial detection relies on pattern recognition. Through history, a range of devices and instruments have been developed to augment and assist human monitoring capability, but the field of sensing (and biosensing) became well established only in the final decades of the 20th century. By way of illustration, in a search in the Web of Science in May, 2021, for published papers containing these terms only a few dozen papers per year were found for the period from 1900 to 1950, most of which unrelated to sensor devices, but rather associated with sensorial phenomena. In the 1970s, an order of magnitude of 1,000 papers published per year was reached, and in the 1990s the annual numbers increased to 10,000 and 20,000, whereas in the last 2 years the number of papers published ranged between 140,000 and 150,000 per year. This outstanding increase in scientific production was obviously a consequence of the progress in research on novel materials, including manmade as well as natural materials adapted for sensing purposes. Though Nature provided inspiration from the early stages, particularly with sensing in animals, two reasons contributed to the field expanding as if entirely independent of human (or animal) sensing. First, real-time monitoring was unfeasible in many sensing applications, and integration of different types of sensors-e.g., to emulate the five senses-remained mostly elusive. Another significant distinction lies in the underlying detection principles: unlike the natural sensors in living beings, normally the analysis of data from manmade sensors does not rely on pattern recognition methods.
The significant progresses in analytical techniques and in sensors and biosensors as nanotech products introduce opportunities for bridging these gaps. Cheap sensors are now routinely fabricated that permit ubiquitous sensing and real-time monitoring in specific applications. Some of these sensors can be wearable and even implantable, and may be more sensitive than their corresponding sensors in humans. An archetypical example is that of an electronic tongue (e-tongue), whose sensitivity for some tastes can be as high as 10,000 times the average sensitivity value for humans (Riul Jr. et al., 2010). Sensing and biosensing employ distinct detection principles, which permits integrating different types of data (e.g., obtained with electronic tongues and electronic noses). Furthermore, computational methods are now available to process the large amounts of data generated with ubiquitous sensing and real-time monitoring, with the bonus of a substantially expanded capacity in terms of memory and processing power, compared to living beings. These methods also allow exploiting pattern recognition strategies, thus bringing the sensing tasks somewhat closer to how sensing is performed by humans. Last, but not least, sensing principles are much broader than those prevailing in living beings, which may open novel ways to fabricate intelligent systems and robots with unprecedented capabilities.
In this paper we focus on two aspects: 1) sensing systems that mimic human senses to exemplify how manmade sensors are being developed with bioinspiration; 2) sensors and biosensors aimed at integration into intelligent systems, in which a discussion will be presented of the stringent requirements that must be fulfilled for data analysis based on computational methods, especially machine learning.

MIMICKING THE FIVE SENSES
Today's technology allows for mimicking the five human senses (Guerrini et al., 2017) with the multiple methodologies discussed below and briefly illustrated with one or two examples, as it is not our purpose to review the major contributions in any of these areas. We shall also distinguish between sensing systems conceived as artificial counterparts of human organs and those simply performing similar functions, even if their shape and nature have nothing to do with the sensing organs.

Touch
Pressure and strain sensors have been developed toward creating electronic skins (Lipomi et al., 2011;Hammock et al., 2013), ionic skins (Qiu et al., 2021) or epidermal electronics (Kim et al., 2011). The terms used may vary and so do the functions performed by e-skins or ionic skins, which can go well beyond those of a human (or other animals) skin. The e-skins share nevertheless the following features: they should be flexible, stretchable, selfhealing, and possess the ability to sense temperature, and wide ranges of pressures (not only touch) and strain. With recent developments in nanomaterials and self-healable polymers, it has been possible to obtain e-skins with augmented performance in comparison to their organic counterparts, especially in superior spatial resolution and thermal sensitivity (Hammock et al., 2013). Future improvements are focused on adding functionalities for specific purposes. For health applications, for instance, biosensors may be incorporated to monitor body conditions and detect diseases, while antimicrobial coatings may be employed to functionalize the e-skins to assist in wound healing (Yang et al., 2019). Within the paradigm of epidermal electronics, on the other hand, the systems envisaged may include not only sensors, but also transistors, capacitors, light-emitting diodes, photovoltaic devices and wireless coils (Kim et al., 2011). Selfpowered tactile sensors produced with piezoelectric polymer nanofibers are indicative of these capabilities (Liu et al., 2021a). An example of the sensing ability of self-healing hydrogels is in detecting distinct body movements, including from speaking which can be relevant for speech processing in the future (Liu et al., 2021b). We shall return to this point when discussing the hearing sense.

Taste
Taste sensing has been mimicked for decades with electronic tongues (e-tongues), which normally contain an array of sensing units with principles of detection based mostly on electrochemical methods (Winquist, 2008) and impedance spectroscopy (Riul et al., 2002). The rationale behind an e-tongue is that humans perceive taste as a combination of five basic tastes, viz. sweet, salty, sour, bitter and umami, thus meaning that the brain receives from the sensors in the papillae signals that are not specific to any given chemical compound. This is the so-called global selectivity principle (Riul et al., 2002), according to which the sensing units, in contrast to biosensors, do not need to contain materials with specific interactions with the samples. Obviously, obtaining sensing units with high sensitivity require their manufacturing materials to be judiciously chosen, bearing in mind the intended application. For example, if an e-tongue is to be used to distinguish liquids with varied acidity levels, polyanilines can be selected for the sensing units since their electrical properties are very sensitive to the pH. In practice, an e-tongue typically comprises four to six sensing units made of nanomaterials or nanostructured polymer films, where the distinct sensing units are expected to yield different responses for a given liquid. This variability is important to establish a "finger print" for the liquids under analysis, which may have similar properties. An e-tongue may take different shapes. While the majority comprise sensing units with nanostructured films deposited over areas of the order of cm 2 , microfluidic e-tongues have also been produced . This is an advantageous arrangement because it requires small amounts of samples for the measurements and allows for multiplex sensing in miniaturized setups.
Though conceived to mimic the tasting function, e-tongues may also be employed in several tasks unrelated to taste. Hence, in addition to their use in evaluating taste in wines, juices, coffee (Riul Jr et al., 2010), taste masking in pharmaceutical drugs (Machado et al., 2018), e-tongues have been utilized in detecting poisoning and pollution in waters, fuel adulteration and soil analysis (Braunger et al., 2017). Three other aspects are worth mentioning about e-tongues. The first is related to the incorporation of biosensors as one (or more) of the sensing units in the arrays. The overall selectivity can be enhanced in these socalled bioelectronic tongues, as demonstrated for the discrimination of two similar tropical diseases (Perinoto et al., 2010). A second aspect refers to data analysis since the use of a global selectivity concept requires assessing the combined responses of various sensing units. As a consequence, a considerable amount of data configurations is generated which must be analyzed with statistical and computational methods. Reduction in the dimensionality of the data representation is thus a central operation. The methods often applied to e-tongue data include principal component analysis (PCA) (Jolliffe and Cadima, 2016) and interactive document mapping (IDMAP) (Minghim et al., 2006). With these methods, the response measured for one sample-e.g., one impedance spectrum-is mapped as a graphical marker, and markers are spatialized so that those markers depicting samples with similar responses will be placed close to each other. Hence, one may identify visual clusters of similar elements, which would suggest a correct classification of the samples is possible in case one univocal cluster exists for each class. If the number of samples is too large, the visualization of clusters on a map is not efficient due to overlapping of markers and clusters. The data may still be processed with machine learning algorithms (Neto et al., 2021), which can be either supervised or unsupervised. In supervised learning, it may be also possible to correlate the e-tongue response with human taste (Ferreira et al., 2007). The third aspect is associated with the combination of e-tongues and electronic noses (e-noses, described below). Especially for drinks and beverages such as coffee and wine, flavor perception depends on taste and smell combined, and therefore it is advisable to employ e-tongues in conjunction with e-noses (Rodriguez-Mendez et al., 2014).

Smell
Electronic noses (e-noses) are the counterparts of e-tongues for smell, being also based on global selectivity concepts where arrays of vapour-sensing devices are employed to mimic the mammalian olfactory system (Rakow and Suslick, 2000). Similar to e-tongues, varied principles of detection can be exploited, including electrical, electrochemical, and optical measurements, or any type of measurement used in gas sensors. The materials for building the sensing units are selected to allow for interaction with various types of vapours, as with metalloporphyrin dyes whose optical properties are affected significantly by ligating vapours such as alcohols, amines, ethers, phosphines and thiols (Rakow and Suslick, 2000). The e-noses are mostly obtained with nanostructured films, e.g., Langmuir-Blodgett (LB) (Barker et al., 1994) and, as in biosensor, may include sensing units capable of specific interaction with analytes. An example of the latter was an e-nose with field-effect transistors made with carbon nanotubes functionalized with lipid nanodiscs containing insect odorant receptors (Murugathas et al., 2019). Their selective electrical response to the corresponding ligands for the odorant receptors allowed for distinguishing the smells from fresh and rotten fish (Murugathas et al., 2019).

Sight
Spectacular developments have been witnessed in computational vision owing to the widespread deployment of all sorts of cameras. Though one may argue that the availability of highquality cameras is far from sufficient for mimicking the sight sense, recent breakthroughs in the field of computational vision demonstrated that artificial systems can already equal, or even surpass the human ability in executing certain image or video analysis tasks (Ng et al., 2018). Facial recognition, for example, can certainly be performed with superior performance by intelligent systems employing deep learning strategies (NandhiniAbirami et al., 2021). Also, cameras may be used to detect and observe phenomena beyond human capabilities, as in the case of infrared vision (Havens and Sharp, 2016). The challenge of replicating the functionality of the human eye in a single device is, nevertheless, still formidable, especially if prosthetic eyes are desired, since a fully functional analogue of the eye remains a long-term goal (Regal et al., 2021). Research on novel materials for bionic eyes and special cameras focuses essentially on bioinspired and biointegrated electronics to fabricate deformable and self-healable devices that preserve functionality while being deformed (Lee et al., 2021). This is analog to other types of devices to mimic human senses, e.g. electronic skin and electronic ear (see below). Another feature shared by these mimicking systems is the need to process large amounts of data, in many cases of entirely different natures. A recent example is represented by an evaluation of withering of tea leaves upon combining data from near-infrared spectroscopy, an electronic eye, and a colorimetric sensing array, which were treated with machine learning algorithms in a machine vision system (Wang et al., 2021).

Hearing
The hearing ability may be mimicked with the so-called electronic ears (Solanki et al., 2017) and other devices, which basically work with strain or pressure sensors. The simplest ones are chemiresistive sensors containing nanomaterials, whose response varies with the mechanical stimulus with sufficient sensitivity to detect whistling, breathing and speaking (Solanki et al., 2017). In terms of materials for sensing, as already mentioned they are similar to those employed for the touch sensors and electronic skins. One issue yet not exploited to a reasonable extent is speech processing. Current applications involving speech processing-prevailing in intelligent assistants, for example-use high-quality microphones to acquire sound. The new developments in wearable strain and pressure sensors allow us to envisage sound acquisition directly from the human (or other living being) body. This would revolutionize the working principles of speech processing, especially in the biomedical area as it would enable real-time online monitoring. Obviously, such an intrusive data collection approach raises ethical issues, for any type of user utterance would be captured and recorded. Another thread is represented by innovative applications made possible by the wide availability of low-cost autonomous microphones, such as the study of environmental soundscapes, for purposes that go from monitoring biodiversity in natural environments (bioacoustics monitoring) (Gibb et al., 2019) or the ocean fauna (Sánchez-Gendriz and Padovese, 2017).

INTEGRATION OF SENSING AND BIOSENSING INTO INTELLIGENT SYSTEMS
Some of the examples associated with the five human senses are already representative of integrated systems based on artificial intelligence. It is relevant, however, that well beyond these examples many other types of sensors and biosensors exist which allow for monitoring substances, phenomena, and processes. In spite of the advances in integrating sensors and biosensors, as mentioned here, we should stress that real-time monitoring and seamless integration of multiple sensing devices using different technologies are still in an embryonic stage, as discussed next with a hypothetical scenario of an autonomous transportation station in a metropolitan area.

Autonomous Public Spaces and Infrastructures
Intelligent systems supported by sensing and biosensing are likely to be employed in any type of application involving control and actuation. Particularly challenging will be such integration in large infrastructures, for instance in public spaces and combining multiple initiatives. As an illustration, let us consider a station serving a busy town area, integrating train and bus services, plus a parking lot, as illustrated in Figure 1. Let us imagine it is connected to a large green park area close to a river, with plenty of vegetation, pedestrian and cycling lanes. City administrators want this area to be a safe zone for users of the station and park. The station should be environmentally sustainable, and they also plan to partner with other city managers to run a preventive health care program targeted at FIGURE 1 | A busy urban transportation station and the multiple sensors supporting its automated operation integrated with multiple initiatives of public interest, that continuously generate data of diverse types.
Frontiers in Sensors | www.frontiersin.org September 2021 | Volume 2 | Article 752754 the population of users and with researchers from the local university to study and preserve the fauna in the park. One may think of the sensing devices deployed in such a scenario. There will be video cameras for real time monitoring for purposes of adjusting train timetables according to the population flow and also of critical areas and spots for security, water and air quality sensors in the station and in the park and surroundings. The site may include a photovoltaic plant, for which energy generation by the plant and energy consumption both in the station and the park areas are continuously tracked and adjusted as necessary. Light-sensitive sensors can switch illumination on/off; sound sensors at multiple spots in the park monitor animal diversity and how the operation of the busy station affects their behavior. Station users are encouraged to stop and collect fundamental health indicators such as blood pressure, glycemic and cholesterol levels, and oriented towards medical assistance when issues are identified; the program may keep track of and approach those users for whom critical issues have been identified. A hypothetical integrated AI system to manage the station autonomously is depicted in the flow chart in Figure 2, which brings an oversimplified abstract view of an intelligent data processing approach. The top layer represents the data sources, including the multitude of sensing devices to monitor internal and external risks in the facilities. These sensors will be continuously generating data of a variety of types, e.g., images, audios, measurements, text forms and documents. It must handle multiple data types, as the data is produced by sensing devices that will include those related to the human senses, i.e., part of the sensing will be sight, touch, taste, smell, and hearing. The diverse data types will demand treatment (storage, filtering, processing) and curation, as indicated the middle block in the figure. Furthermore, some processes will be required to verify if the data makes sense, whether the datasets have sufficient quality as input information for the AI system, represented as a single block for simplicity-though more likely it would consist of multiple integrated systems. Finally, the system will need to learn data representations for algorithmic processing. This AI system will be responsible for the analysis (what happened, where, which is the danger or threat level), then a corresponding action-in some cases in real time. Analysis tasks essentially consist of looking for specific patterns in the data indicating the occurrence of an anomalous situation, or a particular category of event, which in turn must trigger the corresponding actions from the different systems represented in the bottom of the figure. Such a complex scenario consists essentially of a combination of devices for monitoring (quantities and processes) and responding (reacting) to the measurements. The nature of the application may change entirely, e.g., we could think of diagnosis in a medical care facility, or the integrated operating room of a smart city, but the core components of such systems are essentially the same. The implication is that sensing must be ubiquitous and is bound to generate huge quantities of data, which can be connected to the Internet, thus enabling tasks and services to be executed and controlled remotely (Alzahrani, 2017).

Existing AI Technology and Major Challenges
The proposal outlined in Figure 2, of a sensor enabled integrated AI system with multiple controls over its environment, may seem a farfetched view of AI applications today. Yet, a careful analysis of its components indicates that existing technologies are already sufficient for implementing most of the tasks. Indeed, the literature is rich with examples of applications representative of all the components in the figure. Automation has been observed in a multitude of tasks for which the input is digital data, such as object detection in images, or natural language translation, or traffic control, with performance levels equivalent or superior to that of human operators. Smart integration of the required components is, however, a mammoth endeavor. For example, the management and curation of the data from such disparate natures and formats (i.e., images, text, audio, videos, sensing data) is a tremendous challenge. The architecture of the AI system will be highly complex as it must be prepared to detecting and handling multiple ordinary and anomalous situations timely. Even more relevant is that such a system, despite its complexity, is limited in that only classification tasks will be performed efficiently, as already mentioned. Nonetheless, full autonomous operation demands other relevant tasks, such as assessing risks and making decisions based on such assessments. These latter tasks will require at least some degree of interpretation, and therefore current technologies are still not sufficient for the autonomous operation envisaged.
The various issues involved in dealing with big data and machine learning for applications such as the autonomous station have been discussed in reviews and opinion papers (Oliveira et al., 2014;Rodrigues et al., 2016;Paulovich et al., 2018;Rodrigues et al., 2021). We observe a synergistic movement driven by the combination of data generation at unprecedented levels of detail, variety and velocity with massive computing capability. This is crucial to introduce some kind of "intelligence" into systems that process data to execute FIGURE 2 | Schematic representation of the automated system to manage the train station, which should contain various layers of devices and control systems. The top layer represents the varied data sources, whose data will be acquired and curated in dedicated repositories. These will feed the AI system responsible for processing the data and providing the input for the various applications (surveillance, energy management, etc). The AI module and these applications will comprise a number of independent analysis and control systems.
Frontiers in Sensors | www.frontiersin.org September 2021 | Volume 2 | Article 752754 complex tasks. For instance, data now plays an "active" role in science discovery, meaning that rather than solely supporting hypothesis verification, data collected at massive scales with ubiquitous and networked sensing devices connected to "things" (the "Internet of Things") can support "intelligent" automation of complex tasks and foster active search for hidden hypotheses. It must be stressed that building complex autonomous systems as the hypothetical case of the transportation station will demand considerable human effort. Human experts need to be prepared to inspect a system´s underlying algorithms and supervise task execution and decision making during development and operation, in order to ensure it complies with the intended goals. This is different than just inspecting results yielded by a standalone algorithm on a relatively small data set. ML algorithms can be trained to identify patterns from data, but data quality is a fundamental issue (otherwise, "garbage-in, garbage-out"). Moreover, ensuring quality and correctness of the outcomes may demand considerable user supervision. The complexity posed by the sheer scale of data generation and processing, plus the need to handle distinct data types produced by multiple sources in an integrated manner, requires substantial changes in the role of the experts. It also modifies the type of expertise required. Help can be found from researchers working in the field known as "visual analytics" (Endert et al., 2017), in which the goal is to study ways of introducing effective user interaction into data processing and model learning. An additional concern addressed with visual representations is to enhance model interpretability. Domain experts in charge of analysis, as well as decision makers, must be well informed on the concepts behind the techniques, understand their limitations and learn how to parametrize algorithms properly and how to interpret the results and the performance measures. Techniques for data and model visualization can contribute to empowering the mutual roles of a human expert and a ML algorithm in conducting analysis tasks.

CONCLUDING REMARKS
The integration of sensing and biosensing with machine learning algorithms to develop sophisticated autonomous systems has been explored here using two metaphors. In the first, we highlighted recent developments in nanotechnology to fabricate devices that can mimic the five human senses. The motivation behind this choice was not only because these nanotech devices constitute an essential requirement for autonomous entities, but also due to the data processing involved. The other metaphor was related to an autonomous transportation station where we illustrated how an integrated "intelligent" system could manage the station using the input data from a multitude of sensors. This latter example was intended as a mere illustration, for there is a virtually endless list of complex problems could be similarly tackled at a large scale with systems directly fed with data from sensors, with limited human intervention. These range from traffic control to population health and environmental monitoring, precision agriculture to manufacturing processes, amongst others. The wider dissemination of solutions based on machine learning from data provided by sensors is an inescapable trend. Their successful application depends less on technological issues and more on tackling important conceptual and practical ethical, legal and social issues regarding data collection and usage. Yet, some technological issues remain, such as seamless integration of data from different systems and ensuring continuously adaptive learning in highly dynamic environments.

DATA AVAILABILITY STATEMENT
The original contributions presented in the study are included in the article/Supplementary Material, further inquiries can be directed to the corresponding author.