Pressure and strain sensors have been developed toward creating electronic skins (Lipomi et al., 2011; Hammock et al., 2013), ionic skins (Qiu et al., 2021) or epidermal electronics (Kim et al., 2011). The terms used may vary and so do the functions performed by e-skins or ionic skins, which can go well beyond those of a human (or other animals) skin. The e-skins share nevertheless the following features: they should be flexible, stretchable, self-healing, and possess the ability to sense temperature, and wide ranges of pressures (not only touch) and strain. With recent developments in nanomaterials and self-healable polymers, it has been possible to obtain e-skins with augmented performance in comparison to their organic counterparts, especially in superior spatial resolution and thermal sensitivity (Hammock et al., 2013). Future improvements are focused on adding functionalities for specific purposes. For health applications, for instance, biosensors may be incorporated to monitor body conditions and detect diseases, while antimicrobial coatings may be employed to functionalize the e-skins to assist in wound healing (Yang et al., 2019). Within the paradigm of epidermal electronics, on the other hand, the systems envisaged may include not only sensors, but also transistors, capacitors, light-emitting diodes, photovoltaic devices and wireless coils (Kim et al., 2011). Self-powered tactile sensors produced with piezoelectric polymer nanofibers are indicative of these capabilities (Liu et al., 2021a). An example of the sensing ability of self-healing hydrogels is in detecting distinct body movements, including from speaking which can be relevant for speech processing in the future (Liu et al., 2021b). We shall return to this point when discussing the hearing sense.

Taste sensing has been mimicked for decades with electronic tongues (e-tongues), which normally contain an array of sensing units with principles of detection based mostly on electrochemical methods (Winquist, 2008) and impedance spectroscopy (Riul et al., 2002). The rationale behind an e-tongue is that humans perceive taste as a combination of five basic tastes, viz. sweet, salty, sour, bitter and umami, thus meaning that the brain receives from the sensors in the papillae signals that are not specific to any given chemical compound. This is the so-called global selectivity principle (Riul et al., 2002), according to which the sensing units, in contrast to biosensors, do not need to contain materials with specific interactions with the samples. Obviously, obtaining sensing units with high sensitivity require their manufacturing materials to be judiciously chosen, bearing in mind the intended application. For example, if an e-tongue is to be used to distinguish liquids with varied acidity levels, polyanilines can be selected for the sensing units since their electrical properties are very sensitive to the pH. In practice, an e-tongue typically comprises four to six sensing units made of nanomaterials or nanostructured polymer films, where the distinct sensing units are expected to yield different responses for a given liquid. This variability is important to establish a “finger print” for the liquids under analysis, which may have similar properties. An e-tongue may take different shapes. While the majority comprise sensing units with nanostructured films deposited over areas of the order of cm², microfluidic e-tongues have also been produced (Shimizu et al., 2017). This is an advantageous arrangement because it requires small amounts of samples for the measurements and allows for multiplex sensing in miniaturized setups.

Though conceived to mimic the tasting function, e-tongues may also be employed in several tasks unrelated to taste. Hence, in addition to their use in evaluating taste in wines, juices, coffee (Riul Jr et al., 2010), taste masking in pharmaceutical drugs (Machado et al., 2018), e-tongues have been utilized in detecting poisoning and pollution in waters, fuel adulteration and soil analysis (Braunger et al., 2017). Three other aspects are worth mentioning about e-tongues. The first is related to the incorporation of biosensors as one (or more) of the sensing units in the arrays. The overall selectivity can be enhanced in these so-called bioelectronic tongues, as demonstrated for the discrimination of two similar tropical diseases (Perinoto et al., 2010). A second aspect refers to data analysis since the use of a global selectivity concept requires assessing the combined responses of various sensing units. As a consequence, a considerable amount of data configurations is generated which must be analyzed with statistical and computational methods. Reduction in the dimensionality of the data representation is thus a central operation. The methods often applied to e-tongue data include principal component analysis (PCA) (Jolliffe and Cadima, 2016) and interactive document mapping (IDMAP) (Minghim et al., 2006). With these methods, the response measured for one sample—e.g., one impedance spectrum—is mapped as a graphical marker, and markers are spatialized so that those markers depicting samples with similar responses will be placed close to each other. Hence, one may identify visual clusters of similar elements, which would suggest a correct classification of the samples is possible in case one univocal cluster exists for each class. If the number of samples is too large, the visualization of clusters on a map is not efficient due to overlapping of markers and clusters. The data may still be processed with machine learning algorithms (Neto et al., 2021), which can be either supervised or unsupervised. In supervised learning, it may be also possible to correlate the e-tongue response with human taste (Ferreira et al., 2007). The third aspect is associated with the combination of e-tongues and electronic noses (e-noses, described below). Especially for drinks and beverages such as coffee and wine, flavor perception depends on taste and smell combined, and therefore it is advisable to employ e-tongues in conjunction with e-noses (Rodriguez-Mendez et al., 2014).

Electronic noses (e-noses) are the counterparts of e-tongues for smell, being also based on global selectivity concepts where arrays of vapour-sensing devices are employed to mimic the mammalian olfactory system (Rakow and Suslick, 2000). Similar to e-tongues, varied principles of detection can be exploited, including electrical, electrochemical, and optical measurements, or any type of measurement used in gas sensors. The materials for building the sensing units are selected to allow for interaction with various types of vapours, as with metalloporphyrin dyes whose optical properties are affected significantly by ligating vapours such as alcohols, amines, ethers, phosphines and thiols (Rakow and Suslick, 2000). The e-noses are mostly obtained with nanostructured films, e.g., Langmuir-Blodgett (LB) (Barker et al., 1994) and, as in biosensor, may include sensing units capable of specific interaction with analytes. An example of the latter was an e-nose with field-effect transistors made with carbon nanotubes functionalized with lipid nanodiscs containing insect odorant receptors (Murugathas et al., 2019). Their selective electrical response to the corresponding ligands for the odorant receptors allowed for distinguishing the smells from fresh and rotten fish (Murugathas et al., 2019).

Spectacular developments have been witnessed in computational vision owing to the widespread deployment of all sorts of cameras. Though one may argue that the availability of high-quality cameras is far from sufficient for mimicking the sight sense, recent breakthroughs in the field of computational vision demonstrated that artificial systems can already equal, or even surpass the human ability in executing certain image or video analysis tasks (Ng et al., 2018). Facial recognition, for example, can certainly be performed with superior performance by intelligent systems employing deep learning strategies (NandhiniAbirami et al., 2021). Also, cameras may be used to detect and observe phenomena beyond human capabilities, as in the case of infrared vision (Havens and Sharp, 2016). The challenge of replicating the functionality of the human eye in a single device is, nevertheless, still formidable, especially if prosthetic eyes are desired, since a fully functional analogue of the eye remains a long-term goal (Regal et al., 2021). Research on novel materials for bionic eyes and special cameras focuses essentially on bioinspired and biointegrated electronics to fabricate deformable and self-healable devices that preserve functionality while being deformed (Lee et al., 2021). This is analog to other types of devices to mimic human senses, e.g. electronic skin and electronic ear (see below). Another feature shared by these mimicking systems is the need to process large amounts of data, in many cases of entirely different natures. A recent example is represented by an evaluation of withering of tea leaves upon combining data from near-infrared spectroscopy, an electronic eye, and a colorimetric sensing array, which were treated with machine learning algorithms in a machine vision system (Wang et al., 2021).

The hearing ability may be mimicked with the so-called electronic ears (Solanki et al., 2017) and other devices, which basically work with strain or pressure sensors. The simplest ones are chemiresistive sensors containing nanomaterials, whose response varies with the mechanical stimulus with sufficient sensitivity to detect whistling, breathing and speaking (Solanki et al., 2017). In terms of materials for sensing, as already mentioned they are similar to those employed for the touch sensors and electronic skins. One issue yet not exploited to a reasonable extent is speech processing. Current applications involving speech processing—prevailing in intelligent assistants, for example—use high-quality microphones to acquire sound. The new developments in wearable strain and pressure sensors allow us to envisage sound acquisition directly from the human (or other living being) body. This would revolutionize the working principles of speech processing, especially in the biomedical area as it would enable real-time online monitoring. Obviously, such an intrusive data collection approach raises ethical issues, for any type of user utterance would be captured and recorded. Another thread is represented by innovative applications made possible by the wide availability of low-cost autonomous microphones, such as the study of environmental soundscapes, for purposes that go from monitoring biodiversity in natural environments (bioacoustics monitoring) (Gibb et al., 2019) or the ocean fauna (Sánchez-Gendriz and Padovese, 2017).

Intelligent systems supported by sensing and biosensing are likely to be employed in any type of application involving control and actuation. Particularly challenging will be such integration in large infrastructures, for instance in public spaces and combining multiple initiatives. As an illustration, let us consider a station serving a busy town area, integrating train and bus services, plus a parking lot, as illustrated in Figure 1. Let us imagine it is connected to a large green park area close to a river, with plenty of vegetation, pedestrian and cycling lanes. City administrators want this area to be a safe zone for users of the station and park. The station should be environmentally sustainable, and they also plan to partner with other city managers to run a preventive health care program targeted at the population of users and with researchers from the local university to study and preserve the fauna in the park. One may think of the sensing devices deployed in such a scenario. There will be video cameras for real time monitoring for purposes of adjusting train timetables according to the population flow and also of critical areas and spots for security, water and air quality sensors in the station and in the park and surroundings. The site may include a photovoltaic plant, for which energy generation by the plant and energy consumption both in the station and the park areas are continuously tracked and adjusted as necessary. Light-sensitive sensors can switch illumination on/off; sound sensors at multiple spots in the park monitor animal diversity and how the operation of the busy station affects their behavior. Station users are encouraged to stop and collect fundamental health indicators such as blood pressure, glycemic and cholesterol levels, and oriented towards medical assistance when issues are identified; the program may keep track of and approach those users for whom critical issues have been identified.

A hypothetical integrated AI system to manage the station autonomously is depicted in the flow chart in Figure 2, which brings an oversimplified abstract view of an intelligent data processing approach. The top layer represents the data sources, including the multitude of sensing devices to monitor internal and external risks in the facilities. These sensors will be continuously generating data of a variety of types, e.g., images, audios, measurements, text forms and documents. It must handle multiple data types, as the data is produced by sensing devices that will include those related to the human senses, i.e., part of the sensing will be sight, touch, taste, smell, and hearing. The diverse data types will demand treatment (storage, filtering, processing) and curation, as indicated the middle block in the figure. Furthermore, some processes will be required to verify if the data makes sense, whether the datasets have sufficient quality as input information for the AI system, represented as a single block for simplicity—though more likely it would consist of multiple integrated systems. Finally, the system will need to learn data representations for algorithmic processing. This AI system will be responsible for the analysis (what happened, where, which is the danger or threat level), then a corresponding action—in some cases in real time. Analysis tasks essentially consist of looking for specific patterns in the data indicating the occurrence of an anomalous situation, or a particular category of event, which in turn must trigger the corresponding actions from the different systems represented in the bottom of the figure. Such a complex scenario consists essentially of a combination of devices for monitoring (quantities and processes) and responding (reacting) to the measurements. The nature of the application may change entirely, e.g., we could think of diagnosis in a medical care facility, or the integrated operating room of a smart city, but the core components of such systems are essentially the same. The implication is that sensing must be ubiquitous and is bound to generate huge quantities of data, which can be connected to the Internet, thus enabling tasks and services to be executed and controlled remotely (Alzahrani, 2017).

The proposal outlined in Figure 2, of a sensor enabled integrated AI system with multiple controls over its environment, may seem a far-fetched view of AI applications today. Yet, a careful analysis of its components indicates that existing technologies are already sufficient for implementing most of the tasks. Indeed, the literature is rich with examples of applications representative of all the components in the figure. Automation has been observed in a multitude of tasks for which the input is digital data, such as object detection in images, or natural language translation, or traffic control, with performance levels equivalent or superior to that of human operators. Smart integration of the required components is, however, a mammoth endeavor. For example, the management and curation of the data from such disparate natures and formats (i.e., images, text, audio, videos, sensing data) is a tremendous challenge. The architecture of the AI system will be highly complex as it must be prepared to detecting and handling multiple ordinary and anomalous situations timely. Even more relevant is that such a system, despite its complexity, is limited in that only classification tasks will be performed efficiently, as already mentioned. Nonetheless, full autonomous operation demands other relevant tasks, such as assessing risks and making decisions based on such assessments. These latter tasks will require at least some degree of interpretation, and therefore current technologies are still not sufficient for the autonomous operation envisaged.

The various issues involved in dealing with big data and machine learning for applications such as the autonomous station have been discussed in reviews and opinion papers (Oliveira et al., 2014; Rodrigues et al., 2016; Paulovich et al., 2018; Rodrigues et al., 2021). We observe a synergistic movement driven by the combination of data generation at unprecedented levels of detail, variety and velocity with massive computing capability. This is crucial to introduce some kind of “intelligence” into systems that process data to execute complex tasks. For instance, data now plays an “active” role in science discovery, meaning that rather than solely supporting hypothesis verification, data collected at massive scales with ubiquitous and networked sensing devices connected to “things” (the “Internet of Things”) can support “intelligent” automation of complex tasks and foster active search for hidden hypotheses.

It must be stressed that building complex autonomous systems as the hypothetical case of the transportation station will demand considerable human effort. Human experts need to be prepared to inspect a system´s underlying algorithms and supervise task execution and decision making during development and operation, in order to ensure it complies with the intended goals. This is different than just inspecting results yielded by a standalone algorithm on a relatively small data set. ML algorithms can be trained to identify patterns from data, but data quality is a fundamental issue (otherwise, “garbage-in, garbage-out”). Moreover, ensuring quality and correctness of the outcomes may demand considerable user supervision. The complexity posed by the sheer scale of data generation and processing, plus the need to handle distinct data types produced by multiple sources in an integrated manner, requires substantial changes in the role of the experts. It also modifies the type of expertise required. Help can be found from researchers working in the field known as “visual analytics” (Endert et al., 2017), in which the goal is to study ways of introducing effective user interaction into data processing and model learning. An additional concern addressed with visual representations is to enhance model interpretability. Domain experts in charge of analysis, as well as decision makers, must be well informed on the concepts behind the techniques, understand their limitations and learn how to parametrize algorithms properly and how to interpret the results and the performance measures. Techniques for data and model visualization can contribute to empowering the mutual roles of a human expert and a ML algorithm in conducting analysis tasks.