The Use of Artificial Intelligence in Assessing Affective States in Livestock
Suresh Neethirajan- Farmworx, Animal Sciences Department, Wageningen University & Research, Wageningen, Netherlands
In order to promote the welfare of farm animals, there is a need to be able to recognize, register and monitor their affective states. Numerous studies show that just like humans, non-human animals are able to feel pain, fear and joy amongst other emotions, too. While behaviorally testing individual animals to identify positive or negative states is a time and labor consuming task to complete, artificial intelligence and machine learning open up a whole new field of science to automatize emotion recognition in production animals. By using sensors and monitoring indirect measures of changes in affective states, self-learning computational mechanisms will allow an effective categorization of emotions and consequently can help farmers to respond accordingly. Not only will this possibility be an efficient method to improve animal welfare, but early detection of stress and fear can also improve productivity and reduce the need for veterinary assistance on the farm. Whereas affective computing in human research has received increasing attention, the knowledge gained on human emotions is yet to be applied to non-human animals. Therefore, a multidisciplinary approach should be taken to combine fields such as affective computing, bioengineering and applied ethology in order to address the current theoretical and practical obstacles that are yet to be overcome.
Introduction
Compiling evidence that non-human animals are able to use complex cognitive process and show emotions is appearing. An increasing body of literature has investigated the complexity of the animal mind and their ability to show emotions such as fear, pleasure, jealousy and grief (1–4). This evidence leads us to the question of how we could recognize and monitor affective states in livestock. This knowledge is essential to improve the welfare of farm animals kept in captivity through recognizing negative emotions and acting upon them, while promoting positive affective states. However, understanding the animal mind and their emotions is complex as we will only ever be able to use indirect measures to infer affective states. Experimentally testing animal emotions requires careful experimental set up and interpretation of data, which is commonly done through elaborate behavioural observations and tests (judgement bias, attention bias, spatial judgement task etc.,) which is both labour and time intensive, and unfeasible to be practiced on a large scale. In addition, the behavioural testing requires extensive animal training, while some individuals may be unable to be trained which then excludes a certain proportion of animals from being assessed. Artificial intelligence, and in particular machine learning and complex algorithms, combined with the frequent collection of indirect measures with sensors that reflect affective states, can provide us with an objective, efficient and automated means of monitoring animal welfare. While affective computing has received more attention in humans due to its widespread implications for e.g. neuromarketing (5), entertainment (5), monitoring mental health (6, 7) and human-robot interactions (8), the application of affective computing on farm animal welfare is however in its infancy and therefore, more multidisciplinary and exploratory studies are needed to further develop this highly promising field.
The Importance of Understanding the Animal Mind
Understanding when an animal is in distress, pain or fear is essential to improve animal welfare and allow the most animal-friendly practices within farms. Additionally, and maybe even just as important; animal welfare can only really be optimized once we get a better understanding of what promotes positive emotions. We therefore need to understand what types of environments and practices promote positive affective states and what factors can cause stress and discomfort. Moreover, effective recognition of negative affective states and subsequently decreasing stress by removing the causal factors may increase animal productivity, reduce disease and prevent infection (9–15).
Emotions and Affective States
Before we can understand how to effectively measure emotions in non-human animals, a definition of the concept is required and an understanding of how emotion differs from affective state and mood. An affective state encompasses both emotions, which are short-term and directed at a particular source, and mood, which is less intense than an emotion and can appear without a particular stimulus to cause it (16). Most definitions of emotion originated from the field of psychology, but no consensus has been achieved yet on the concept, nor the number of emotions that exist. One definition that is relevant when we are looking for methods to measure emotion, is the definition by Broom and Johnson which states that an emotion is “a physiologically describable component of a feeling characterized by electrical and neurochemical activity in particular regions of the brain, autonomic nervous system activity, hormone release and peripheral consequences including behaviour”, whereas a feeling also involves the perceptual awareness of this emotion by the individual (17). Affective states are important for welfare because of their subjective “feeling” component, i.e. because they are experienced as pleasant or unpleasant. If there is an emotion without perceptual awareness, then such an emotion may be of no relevance to animal welfare. Izard has proposed emotion schemas as an attempt to scientifically define emotions (18). Emotion schemas consist of “emotion interacting dynamically with cognitive and perceptual processes to influence behaviour and mind” (19). An internal or external stimulus can cause a particular emotion, accompanied by measurable changes that in turn, can lead to a behavioural response to cope with the challenges of the physical and social environment.
Paul Eckman suggested six basic emotions: happiness, sadness, fear, disgust, anger and surprise (20). The criteria that he used to define a basic emotion consist of distinctive facial expressions that are universal, the presence in other primates, a distinctive psychology, a rapid onset, brief duration and that the appraisal of them is automatic (21). Additionally, Robert Plutchik also suggested that emotions should provide a function that aids survival and therefore touched upon the evolutionary significance of emotion (22). Whereas fear is the emotion that has received the greatest attention, compiling evidence suggests that other emotions such as grief, jealousy and happiness are apparent in non-human animals, too (1–4, 23). Even though measuring discrete emotions is an exciting and relatively unexplored field, more commonly a dimensional approach is taken to measure emotion in non-human animals. The main two dimensions measured here are the level of arousal (high/low) and emotional valence (positive vs. negative depending on the level of reward/punishment of the stimulus) (16). Authors such as Ekman and Plutchik have not considered sensory and homeostatic states such as pain, hunger, nausea etc. in the definition of emotion. However, according to Jaak Panksepp, these homeostatic affective states rely on different brain mechanisms and should be considered as a separate category from 'Ekmanian' emotions caused by external events (18). An attempt to create a common definition of emotion to the affective sciences is a fruitless debate (24) due to the differences in disciplines such as psychology, neuroscience, philosophy and animal sciences. Few theories consider emotions as concomitants of both bodily and cognitive elements (25). Due to the reciprocal influence between brain and body systems such as nervous system and immune system, the physiological account of stress can be illustrated as an emotional condition (25). Although stress can be considered as an affective state, it rather fits the category of long-term affective states rather than the category of emotions which are short-term states caused by specific stimuli. The scope of the paper is about affective states (i.e. states with a valenced feeling element) and that this broad category includes sensory and homeostatic affective states, emotions, and moods.
How Can We Measure Affective States in Non-Human Animals?
However, one major obstacle within the field of animal affective states is our inability to accurately measure affective states. Whereas the identification and recognition of affective states in humans is commonly accompanied by questionnaires to validate the effectiveness of the methods in question, getting a confirmation in the same method as humans is impossible. Research on human affective states has focused on the ability to measure emotion based on verbal speech, written text and hand gestures, amongst other measures, which are all means of communication that are not or less used by non-human animals. A great number of behavioural tests have been employed to allow assessment of affective states in other non-human animals, including open field, human interaction, cognitive bias and elevated plus maze tests (26–29). More studies keep appearing that show a correlation between indirect measures and the results of such behavioural tests, and therefore such measures provide a more efficient method of monitoring animal affective state without the need for individual behavioural testing. Such indirect measures include hormonal assessments, physiological measures, facial expression, brain activity, thermal imaging, vocalisations and movement (30). In multiple farm animal species (cattle, sheep, goats, horses, pigs, poultry), emotional valency has been associated with particular vocalisations (31–37), changes in nasal temperature (38) or eye temperature (39–41), cortisol levels (39, 42–44), heart rate and heart rate variability (40, 45–49), respiration rate (50) but also facial expression and the (change in) position of the ears and tail (31, 43, 46, 51–62). See Table 1 for a collection of studies that tested these measures and how they relate to emotional valence. The papers cited in the table were collected using the Web of Science tool. Past 10-year peer reviewed published papers were only considered for inclusion in the table. Keywords for searching in the web of science database included farm animals, pigs, cows, horses, sheep, chickens, emotions, indicators of welfare, and affective states. Boolean and individual searches using the above-mentioned keywords were used for the literature collection.
Table 1. Summary of literature that measures indicators of affective states using unimodal and multimodal approaches in farm animals.
The Need for a Multimodal Approach
These measures all have their advantages and disadvantages. Ideally, the measure that is used as an indicator of affective state will be (1) non-invasive, (2) will produce high-quality data with low sensitivity to environmental disturbances, (3) can be automated to reduce the time and labour required to collect the data, (4) is able to track affective states of individual animals, (5) is not too costly and (6) can identify subtle changes to allow prevention of negative affective state. Naturally, no measure to date is able to check all these boxes, and therefore the main emphasis is on defining a reliable indicator that is consistent across different contexts. One issue with using just a single measure, however, is that a positive and negative emotional valence might have a similar effect on the measure used (38). Furthermore, measures are not always consistent between different studies (67). This highlights the importance of implementing a multimodal approach, combining different measures to get a better understanding of the affective state of the animals. Such an approach has already been emphasized within human affective research (68, 69) as multimodal approaches are able to assess affective states more accurately than any single measure could do on its own. Affective expression consists of complex coordination of signals made of physiology; facial expressions; posture and behavioural indicators. Multimodality signals consists of various channels of expressions such as audio, postures, gestures/activity, visual and eye gage etc. Unimodal signals have the drawback because it is associated with only one type of expression (70). Unimodal affect detectors are also inherently noisy due to the association between affective states and the specific individual signal. The coupling between observable expressions and experience of specific affective states is weaker for unimodal in comparison to the multimodal approach (70). For example: Interactions of auditory and visual emotional information (multi-cues) are not only limited to communication aspects but also to contexts such as environmental cues, contagion aspects and social interactions. Current advancements in human research are moving from traditional unimodal to multimodal approaches in assessing affective states due to the need for enhanced accuracy and the ability to integrate multiple sensory channels. To enable the welfare monitoring and decision support systems in accurately interpreting the affective states of farm animals, multimodal based affective computing is an absolute need. In order to create a reliable framework incorporating multiple factors, the different measures have to be temporally synchronized, and an in-depth knowledge needs to be available about how the measures intercorrelate. The most important limiting factor here is that a large amount of reproducible data needs to be collected to build such a framework.
Self-Learning Algorithms Are the Answer
Artificial intelligence and machine learning can provide an efficient, objective and automatic self-learning mechanism that could classify different affective states based on multiple factors, while using a sensor-based approach to frequently collect data (30, 71–75). Machine learning consists of computer algorithms learning as more data are added to the system, while being able to recognize patterns and/or features and categorize them. Such systems are able to analyse and predict affective states and notify the user of any abnormalities. So, instead of manually analysing video or photo footage, or live observations of animals by often multiple experimenters recording different measures, using sensors will reduce the amount of time and labour required to collect needed data. To allow the system to recognize affective changes, a baseline level of the measures is required to understand animals in a “neutral” environment, and through experimentally testing the animals in a known positive or negative environment, inferences can then be made about affective states changes. Machine learning mechanisms require a large amount of high-quality data which are used to train the model. By using sensors that can continuously collect data, subtle changes are easier to pick up and simultaneously, large amounts of data continue to be collected that can feed the algorithm and improve its accuracy. As more data are collected, the accuracy of subtle changes will improve too, which would allow early detection and therefore prevention of e.g. tension within the group that has the potential to escalate, disease or infection, and other health problems. Such data would have to be collected for each species in investigation, and an interdisciplinary approach is needed combining computational science, animal science and ethology.
Currently, research within affective computing for non-human animals is pioneering. One of the first steps to take is choosing measures that complement each other and collectively give an accurate representation of the animal's affective state. Promising sensors include infrared thermography and facial expression / posture. What sets apart these types of sensors is their ability to recognize individual animals while recognizing and investigating e.g. the temperature of a particular regions of interest, such as the eyes, ears or tails. Moreover, it has already been shown that the equipment needed to collect such non-invasive data can be mobilized, increasing the feasibility to employ such equipment within a larger industrial setting (40, 50, 76). More and more studies demonstrate the applications of automated systems within a farm setting, highlighting the potential of this field, ranging from surveying pig enclosure usage to identifying heat stress (77) to infrared thermography to track cattle health (76). Human affective research is only just starting to understand the importance of multimodal approaches, and this knowledge should be used to develop models for non-human animals, too. Research on humans have resulted in complex systems that allow accurate sentiment analysis, preference detection (5) using qualitative and quantitative data such as facial expressions, body gestures, phonetic and acoustic properties of spoken language, word use and grammar in written text and more (8, 68). Different algorithms have been designed using computational methodologies such as hidden Markov chains, Bayesian networks and Gaussian mixture models to e.g. recognize facial expressions (78) and allow automatic detection of regions of interest such as the nose (79). An end-to-end emotion prediction task using a multimodal system from visual and speech data was developed by Tzirakis et al., 2017 (80). The comparative analysis by testing and validation of the emotion prediction system proved that multimodal model is a better valence dimension assessor than the unimodal model. Utilizing the multimodal functional near-infrared spectroscopy (fNIRS), electroencephalograph (EEG) wearable sensors, and capture of facial expressions through video cameras, Sun et al., 2020 (81) investigated the relationship between multimodal brain activity and spontaneous facial affective expressions. The strength of the emotional valence correlation to affective expressions revealed that the multimodal method outperformed the unimodal approaches individually. The outcomes from this study also emphasized the utilization of facial expression and wearable neuroimaging sensors for affective brain-computer applications.
Limitations and Challenges to Overcome
However, there are multiple theoretical and practical challenges still to overcome. First of all, sensors that provide high-quality multimodal data except for video and thermal infrared cameras are not yet commercially available plus the high costs associated with them might deter farmers to employing such technology. Even though farmers can financially benefit from the early detection of health problems due to a reduction in antibiotic usage and need for veterinary assistance, accompanied by an improvement in welfare of their animals, farmers could be concerned due to data use, misuse and storage and the monetization of data (82). In addition to the financial inability to invest in such technologies, the access to a reliable internet connection and high computing power are essential for the functioning of the algorithms and alerting the farmer timely, which poses a limit to digital farming technologies (74). Most farmers do not have the money to invest in either the research or technology, which means that funding within this field should come from governmental bodies and/or subsidies. Numerous studies have shown that improved welfare in farm animals can positively affect the productivity of the farm and the quality of the animal product. For example, higher levels in serotonin (a hormone associated with positive affective states) can increase calcium levels in dairy cows (14), heat stress in dairy cattle can reduce milk yield and quality (83), stress can reduce meat quality in multiple species (84, 85) and prolonged stress can increase disease risk (12, 13). Ensuring a positive affective state in production animals therefore is not only a benefit to the animals themselves, but also the farmer and consumers. In order for grants to be given, an increase in the public awareness on the complexity of the animal mind, including emotionality, personality, intelligence and cognition should be addressed to facilitate the recognition of the importance of this field.
Other than limitations from the farmer's perspective and funding limitations, the accuracy of the measures is an important topic of discussion as well regarding the practicality of using artificial intelligence for farm animal monitoring. First of all, compiling evidence within ethology and behavioural ecology pinpoints the large inter-individual differences in personality, behaviour and in effect, emotionality (86–89). Such variation will have to be incorporated within the algorithm to avoid creating a detection bias. Secondly, affective computing and applied ethology are two distinct disciplines and a lack of interdisciplinary studies prevents the exchange of valuable information between the fields. An interdisciplinary approach should be taken to combine ethological studies with modern, advanced computing systems and bioengineering approaches to create accurate, efficient, species-specific algorithms fine-tuned to the complexity of the animal's behaviour. Furthermore, even though the field of affective computing in farm animals is rising, many areas remain unexplored. Several sensory systems and means of inter- and intraspecific communication have not been utilized as of yet in order to explore emotionality and emotional contagion between animals. Despite our increasing awareness of the importance of olfaction within animal ecology, such as stress signaling to conspecifics when presented a predator odour (90) or evidence that stress hormones can affect olfactory regulation and perception (91), this area remains unaddressed in the context of affective state recognition. The presence of biomarkers in sweat, such as volatile organic components that are created in stressful environments (92), in combination with scent marks and fecal or urinal excretion of compounds remains an unexplored indicator of affective states. Despite the challenges that olfactory studies bring, such as technical challenges like consistent stimulus delivery, environmental disturbances (wind, other smells, physical factors that affect volatility and spatial distribution), distances to the source of the odour, among others, each dimension that contributes to animal emotionality should be explored, including olfactory factors that might indicate the inner state of the group or individual.
Concluding Remarks
The Implications and usage of artificial intelligence to monitor farm animal affective state are widespread and rising technologies within human studies should be utilized and adapted to broaden the purposes of affective computing. Despite many obstacles that are yet to be overcome, it is important to recognize the problems and goals that each individual farmer might face, in order to tailor smart systems to the needs of the farm and in effect, its animals. Whereas the need for reducing disease, infection or conflict might be higher in some places and/or species, other farmers might desire to adopt even more animal-friendly and humane practices and want to focus on monitoring positive affective states to uplift animal welfare. All these factors will in turn affect the most efficient and effective technological systems and measures taken to address such goals. Public awareness should be boosted on the presence and importance of animal emotionality so that technology not only advances to identify human cyberbullying, utilize neuromarketing and promote affective entertainment, but also improve the quality of life of the billions of domesticated animals that are born and bred in human captivity.
Author Contributions
The author confirms being the sole contributor of this work and has approved it for publication.
Conflict of Interest
The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher's Note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
References
1. Morris PH, Doe C, Godsell E. Secondary emotions in non-primate species? Behavioural reports and subjective claims by animal owners. Cogn Emot. (2008) 22:3–20. doi: 10.1080/02699930701273716
2. Balcombe J. Animal pleasure and its moral significance. Appl Anim Behav Sci. (2009) 118:208–16. doi: 10.1016/j.applanim.2009.02.012
4. Harris CR, Prouvost C. Jealousy in dogs. PLoS ONE. (2014) 9:e94597. doi: 10.1371/journal.pone.0094597
5. Teo J, Hou L, Tian J, Mountstephens J. Classification of affective states via EEG and deep learning. Int J Adv Comput Sci Appl. (2018) 9:132–42. doi: 10.14569/IJACSA.2018.090517
6. Balducci F, Impedovo D, Macchiarulo N, Pirlo G. Affective states recognition through touch dynamics. Multimed Tools Appl. (2020) 79:35909–26. doi: 10.1007/s11042-020-09146-4
7. Adikari A, Gamage G, de Silva D, Mills N, Wong S-MJ, Alahakoon D, et al. self structuring artificial intelligence framework for deep emotions modeling and analysis on the social web. Future Gener Comput Syst. (2021) 116:302–15. doi: 10.1016/j.future.2020.10.028
8. Abbaschian BJ, Sierra-Sosa D, Elmaghraby A. Deep learning techniques for speech emotion recognition, from databases to models. Sensors. (2021) 21:1249. doi: 10.3390/s21041249
9. von Borell E. Neuroendocrine integration of stress and significance of stress for the performance of farm animals. Appl Anim Behav Sci. (1995) 44:219–27. doi: 10.1016/0168-1591(95)00615-Y
10. Rostagno MH. Can stress in farm animals increase food safety risk? Foodborne Pathog Dis. (2009) 6:767–76. doi: 10.1089/fpd.2009.0315
11. Kumar B, Manuja A, Aich P. Stress and its impact on farm animals. Front Biosci. (2012) 4:1759–67. doi: 10.2741/496
12. Etim NN, Williams ME, Evans EI, Offiong EE. Physiological and behavioural responses of farm animals to stress: implications to animal productivity. Am J Adv Agric Res. (2013) 1:53–61.
13. Proudfoot K, Habing G. Social stress as a cause of diseases in farm animals: current knowledge and future directions. Vet J. (2015) 206:15–21. doi: 10.1016/j.tvjl.2015.05.024
14. Weaver SR, Prichard AP, Endres EL, Newhouse SA, Peters TL, Crump PM, et al. Elevation of circulating serotonin improves calcium dynamics in the peripartum dairy cow. J Endocrinol. (2016) 230:105–23. doi: 10.1530/JOE-16-0038
15. Gebregeziabhear E, Ameha N, Zeit D, Dawa D. The effect of stress on productivity of animals: a review. JBAH. (2015) 5:14–22.
16. Mendl M, Burman OHP, Paul ES. An integrative and functional framework for the study of animal emotion and mood. Proc R Soc B Biol Sci. (2010) 277:2895–904. doi: 10.1098/rspb.2010.0303
17. Donald MB. Stress and Animal Welfare: Key Issues in the Biology of Humans and Other Animals. Springer (2020). doi: 10.1007/978-3-030-32153-6
18. Panksepp J. Affective consciousness: Core emotional feelings in animals and humans. Conscious Cogn. (2005) 14:30–80. doi: 10.1016/j.concog.2004.10.004
19. Izard CE. Emotion theory and research: highlights, unanswered questions, and emerging issues. Annu Rev Psychol. (2009) 60:1–25. doi: 10.1146/annurev.psych.60.110707.163539
20. Ekman P. Cross-cultural Studies of Facial Expression. Darwin and Facial Expression: a Century of Research in Review. New York, NY: Academic Press (1973) 169–222.
21. Ekman P, Davidson R. All Emotions are Basic. The Nature of Emotion: Fundamental Questions. New York: Oxford University Press (1994).
23. Kemeny ME, Gruenewald TL, Dickerson SS. Shame as the emotional response to threat to the social self: implications for behavior, physiology, and health. Psychol Inq. (2004) 15:153–60.
24. Mulligan K, Scherer KR. Toward a working definition of emotion. Emot Rev. (2012) 4:345–57. doi: 10.1177/1754073912445818
25. Colombetti G, Zavala E. Are emotional states based in the brain? A critique of affective brainocentrism from a physiological perspective Biol Philos. (2019) 34:45. doi: 10.1007/s10539-019-9699-6
26. Forkman B, Boissy A, Meunier-Salaün M-C, Canali E, Jones RBA. critical review of fear tests used on cattle, pigs, sheep, poultry and horses. Physiol Behav. (2007) 92:340–74. doi: 10.1016/j.physbeh.2007.03.016
27. Murphy E, Nordquist RE, van der Staay FJA. review of behavioural methods to study emotion and mood in pigs, Sus scrofa. Appl Anim Behav Sci. (2014) 159:9–28. doi: 10.1016/j.applanim.2014.08.002
28. Lagisz M, Zidar J, Nakagawa S, Neville V, Sorato E, Paul ES, et al. Optimism, pessimism and judgement bias in animals: a systematic review and meta-analysis. Neurosci Biobehav Rev. (2020) 118:3–17. doi: 10.1016/j.neubiorev.2020.07.012
29. Harding EJ, Paul ES, Mendl M. Cognitive bias and affective state. Nature. (2004) 427:312. doi: 10.1038/427312a
30. Neethirajan S, Reimert I, Kemp B. Measuring farm animal emotions—Sensor-based approaches. Sensors. (2021) 21:553. doi: 10.3390/s21020553
31. Sandem AI, Braastad BO, Bøe KE. Eye white may indicate emotional state on a frustration–contentedness axis in dairy cows. Appl Anim Behav Sci. (2002) 79:1–10. doi: 10.1016/S0168-1591(02)00029-1
32. Reimert I, Bolhuis JE, Kemp B, Rodenburg TB. Indicators of positive and negative emotions and emotional contagion in pigs. Physiol Behav. (2013) 109:42–50. doi: 10.1016/j.physbeh.2012.11.002
33. Du X, Lao F, Teng GA. sound source localisation analytical method for monitoring the abnormal night vocalisations of poultry. Sensors. (2018) 18:2906. doi: 10.3390/s18092906
34. Stomp M, Leroux M, Cellier M, Henry S, Lemasson A, Hausberger M. An unexpected acoustic indicator of positive emotions in horses. PLoS ONE. (2018) 13:e0197898. doi: 10.1371/journal.pone.0197898
35. Friel M, Kunc HP, Griffin K, Asher L, Collins LM. Positive and negative contexts predict duration of pig vocalisations. Sci Rep. (2019) 9:2062. doi: 10.1038/s41598-019-38514-w
36. Herborn KA, McElligott AG, Mitchell MA, Sandilands V, Bradshaw B, Asher L. Spectral entropy of early-life distress calls as an iceberg indicator of chicken welfare. J R Soc Interface. (2020) 17:20200086. doi: 10.1098/rsif.2020.0086
37. Green AC, Lidfors LM, Lomax S, Favaro L, Clark CE. Vocal production in postpartum dairy cows: temporal organization and association with maternal and stress behaviors. J Dairy Sci. (2021) 104:826–38. doi: 10.3168/jds.2020-18891
38. Proctor HS, Carder G. Nasal temperatures in dairy cows are influenced by positive emotional state. Physiol Behav. (2015) 138:340–4. doi: 10.1016/j.physbeh.2014.11.011
39. Valera M, Bartolomé E, Sánchez MJ, Molina A, Cook N, Schaefer A. Changes in eye temperature and stress assessment in horses during show jumping competitions. J Equine Vet Sci. (2012) 32:827–30. doi: 10.1016/j.jevs.2012.03.005
40. Bartolomé E, Sánchez MJ, Molina A, Schaefer AL, Cervantes I, Valera M. Using eye temperature and heart rate for stress assessment in young horses competing in jumping competitions and its possible influence on sport performance. Anim Int J Anim Biosci. (2013) 7:2044–53. doi: 10.1017/S1751731113001626
41. Edgar JL, Nicol CJ, Pugh CA, Paul ES. Surface temperature changes in response to handling in domestic chickens. Physiol Behav. (2013) 119:195–200. doi: 10.1016/j.physbeh.2013.06.020
42. Peeters M, Sulon J, Beckers J-F, Ledoux D, Vandenheede M. Comparison between blood serum and salivary cortisol concentrations in horses using an adrenocorticotropic hormone challenge. Equine Vet J. (2011) 43:487–93. doi: 10.1111/j.2042-3306.2010.00294.x
43. Lansade L, Nowak R, Lainé A-L, Leterrier C, Bonneau C, Parias C, et al. Facial expression and oxytocin as possible markers of positive emotions in horses. Sci Rep. (2018) 8:1–11. doi: 10.1038/s41598-018-32993-z
44. Lürzel S, Bückendorf L, Waiblinger S, Rault J-L. Salivary oxytocin in pigs, cattle, and goats during positive human-animal interactions. Psychoneuroendocrinology. (2020) 115:104636. doi: 10.1016/j.psyneuen.2020.104636
45. Von Borell E, Langbein J, Després G, Hansen S, Leterrier C, Marchant-Forde J, et al. Heart rate variability as a measure of autonomic regulation of cardiac activity for assessing stress and welfare in farm animals—a review. Physiol Behav. (2007) 92:293–316. doi: 10.1016/j.physbeh.2007.01.007
46. König von Borstel U, Duncan I, Shoveller A, Merkies K, Keeling L, Millman S. Impact of riding in a coercively obtained Rollkur posture on welfare and fear of performance horses. Appl Anim Behav Sci. (2009) 116:228–36. doi: 10.1016/j.applanim.2008.10.001
47. Schmied C, Boivin X, Scala S, Waiblinger S. Effect of previous stroking on reactions to a veterinary procedure: behaviour and heart rate of dairy cows. Interact Stud. (2010) 11:467–81. doi: 10.1075/is.11.3.08sch
48. Mott RO, Hawthorne SJ, McBride SD. Blink rate as a measure of stress and attention in the domestic horse (Equus caballus). Sci rep. (2020) 10:1–8. doi: 10.1038/s41598-020-78386-z
49. Ziemiańska A, Kasperek K, Rozempolska-Rucińska I, Zieba G, Czech A. Behaviour and stress in three breeds of laying hens kept in the same environment. S Afr J Anim Sci. (2020) 50:272–80. doi: 10.4314/sajas.v50i2.10
50. Lowe G, Sutherland M, Waas J, Schaefer A, Cox N, Stewart M. Infrared thermography—A non-invasive method of measuring respiration rate in calves. Animals. (2019) 9:535. doi: 10.3390/ani9080535
51. Reefmann N, Bütikofer Kaszàs F, Wechsler B, Gygax L. Ear and tail postures as indicators of emotional valence in sheep. Appl Anim Behav Sci. (2009) 118:199–207. doi: 10.1016/j.applanim.2009.02.013
52. Proctor HS, Carder G. Can ear postures reliably measure the positive emotional state of cows? Appl Anim Behav Sci. (2014) 161:20–7. doi: 10.1016/j.applanim.2014.09.015
53. Proctor HS, Carder G. Measuring positive emotions in cows: do visible eye whites tell us anything? Physiol Behav. (2015) 147:1–6. doi: 10.1016/j.physbeh.2015.04.011
54. Guesgen MJ, Beausoleil NJ, Leach M, Minot EO, Stewart M, Stafford KJ. Coding and quantification of a facial expression for pain in lambs. Behav Processes. (2016) 132:49–56. doi: 10.1016/j.beproc.2016.09.010
55. Hintze S, Smith S, Patt A, Bachmann I, Würbel H. Are eyes a mirror of the soul? What eye wrinkles reveal about a horse's emotional state. PLoS ONE. (2016) 11:e0164017. doi: 10.1371/journal.pone.0164017
56. McLennan KM, Rebelo CJ, Corke MJ, Holmes MA, Leach MC, Constantino-Casas F. Corrigendum to development of a facial expression scale using footrot and mastitis as models of pain in sheep. Appl Anim Behav Sci. (2016) 176:19–26. doi: 10.1016/j.applanim.2016.01.007
57. Lu Y, Mahmoud M, Robinson P. Estimating sheep pain level using facial action unit detection. In: 2017 12th IEEE International Conference on Automatic Face Gesture Recognition (FG 2017) (2017) p. 394–9. doi: 10.1109/FG.2017.56
58. Oliveira D de, Keeling LJ. Routine activities and emotion in the life of dairy cows: Integrating body language into an affective state framework. PLoS ONE. (2018) 13:e0195674. doi: 10.1371/journal.pone.0195674
59. Marcet Rius M, Pageat P, Bienboire-Frosini C, Teruel E, Monneret P, Leclercq J, et al. Tail and ear movements as possible indicators of emotions in pigs. Appl Anim Behav Sci. (2018) 205:14–8. doi: 10.1016/j.applanim.2018.05.012
60. McLennan K, Mahmoud M. Development of an automated pain facial expression detection system for sheep (Ovis Aries). Animals. (2019) 9:196. doi: 10.3390/ani9040196
61. Czycholl I, Hauschild E, Büttner K, Krugmann K, Burfeind O, Krieter J. Tail and ear postures of growing pigs in two different housing conditions. Behav Processes. (2020) 176:104138. doi: 10.1016/j.beproc.2020.104138
62. Mota-Rojas D, Orihuela A, Martínez-Burnes J, Gómez J, Mora-Medina P, Alavez B, et al. Neurological modulation of facial expressions in pigs and implications for production. J Anim Behav Biometeorol. (2020) 8:232–43. doi: 10.31893/jabb.20031
63. López-Arjona M, Mateo S, Manteca X, Escribano D, Cerón J, Martinez-Subiela S. Oxytocin in saliva of pigs: an assay for its measurement and changes after farrowing. Domest Anim Endocrinol. (2019) 70:106384. doi: 10.1016/j.domaniend.2019.106384
64. Battini M, Agostini A. Mattiello S. Understanding cows' emotions on farm: are eye white and ear posture reliable indicators? Animals. (2019) 9:477. doi: 10.3390/ani9080477
65. Gaughan JB, Holt S, Hahn GL, Mader TL, Eigenberg R. Respiration rate—is it a good measure of heat stress in cattle? Asian-Aus J Anim Sci. (2000) 13:329–32.
66. Tarantola M, Biasato I, Biasibetti E, Biagini D, Capra P, Guarda F, et al. Beef cattle welfare assessment: use of resource and animal-based indicators, blood parameters and hair 20β-dihydrocortisol. Ital J Anim Sci. (2020) 19:341–50. doi: 10.1080/1828051X.2020.1743783
67. Gómez Y, Bieler R, Hankele AK, Zähner M, Savary P, Hillmann E. Evaluation of visible eye white and maximum eye temperature as non-invasive indicators of stress in dairy cows. Appl Anim Behav Sci. (2018) 198:1–8. doi: 10.1016/j.applanim.2017.10.001
68. Poria S, Cambria E, Bajpai R, Hussain A. review of affective computing: From unimodal analysis to multimodal fusion. Inf Fusion. (2017) 37:98–125. doi: 10.1016/j.inffus.2017.02.003
69. Shoumy NJ, Ang LM, Seng KP, Rahaman DMM, Zia T. Multimodal big data affective analytics: A comprehensive survey using text, audio, visual and physiological signals. J Netw Comput Appl. (2020) 149:1–26. doi: 10.1016/j.jnca.2019.102447
70. D'mello SK, Kory J. A review and meta-analysis of multimodal affect detection systems. ACM Comput Surv. (2015) 47:13–6. doi: 10.1145/2682899
71. Cihan P, Gökçe E, Kalipsiz O. A review of machine learning applications in veterinary field. Kafkas Univ Vet Fak Derg. (2017) 23:673–80. doi: 10.9775/kvfd.2016.17281
72. Yates H, Chamberlain B, Baldwin W, Hsu WH, Vanlandingham D. Assessing animal emotion and behavior using mobile sensors and affective computing. In: Emerging Trends and Applications in Cognitive Computing (2019) p. 49–77. doi: 10.4018/978-1-5225-5793-7.ch003
73. Budaev S, Kristiansen TS, Giske J, Eliassen S. Computational animal welfare: towards cognitive architecture models of animal sentience, emotion and wellbeing. R Soc Open Sci. (2020) 7:201886. doi: 10.1098/rsos.201886
74. Neethirajan S. The role of sensors, big data and machine learning in modern animal farming. Sens Bio-Sens Res. (2020) 29:100367. doi: 10.1016/j.sbsr.2020.100367
75. Neethirajan S, Kemp B. Social network analysis in farm animals: Sensor-based approaches. Animals. (2021) 11:434. doi: 10.3390/ani11020434
76. Cho Y, Bianchi-Berthouze N, Oliveira M, Holloway C, Julier S. Nose heat: exploring stress-induced nasal thermal variability through mobile thermal imaging. In: 2019 8th International Conference on Affective Computing and Intelligent Interaction (ACII) (2019) p. 566–72. doi: 10.1109/ACII.2019.8925453
77. Nilsson M, Herlin AH, Ardö H, Guzhva O, Åström K, Bergsten C. Development of automatic surveillance of animal behaviour and welfare using image analysis and machine learned segmentation technique. Animal. (2015) 9:1859–65. doi: 10.1017/S1751731115001342
78. Dolensek N, Gogolla N. Machine-learning approaches to classify and understand emotion states in mice. Neuropsychopharmacology. (2020) 46:250–1. doi: 10.1038/s41386-020-00857-8
79. Kalal Z, Mikolajczyk K, Matas J. Forward-backward error: automatic detection of tracking failures. In: 2010 20th International Conference on Pattern Recognition (2010) p. 2756–9. doi: 10.1109/ICPR.2010.675
80. Tzirakis P, Trigeorgis G, Nicolaou MA, Schuller B, Zafeiriou S. End-to-end multimodal emotion recognition using deep neural networks. IEEE J Sel Top Signal Process. (2017) 11:1301–9. doi: 10.1109/JSTSP.2017.2764438
81. Sun Y, Ayaz H, Akansu AN. Multimodal affective state assessment using fNIRS + EEG and spontaneous facial expression. Brain Sci. (2020) 10:85. doi: 10.3390/brainsci10020085
82. Wolfert S, Ge L, Verdouw C, Bogaardt M-J. Big data in smart farming – A review. Agric Syst. (2017) 153:69–80. doi: 10.1016/j.agsy.2017.01.023
83. Pragna P, Archana PR, Aleena J, Sejian V, Krishnan G, Bagath M, et al. Heat stress and dairy cow: Impact on both milk yield and composition. Int J Dairy Sci. (2017) 12:1–11. doi: 10.3923/ijds.2017.1.11
84. Ferguson DM, Warner RD. Have we underestimated the impact of pre-slaughter stress on meat quality in ruminants? Meat Sci. (2008) 80:12–9. doi: 10.1016/j.meatsci.2008.05.004
85. Akşit M, Yalçin S, Özkan S, Metin K, Özdemir D. Effects of temperature during rearing and crating on stress parameters and meat quality of broilers. Poult Sci. (2006) 85:1867–74. doi: 10.1093/ps/85.11.1867
86. Désiré L, Boissy A, Veissier I. Emotions in farm animals: a new approach to animal welfare in applied ethology. Behav Processes. (2002) 60:165–80. doi: 10.1016/S0376-6357(02)00081-5
87. Sih A, Mathot KJ, Moirón M, Montiglio P-O, Wolf M, Dingemanse NJ. Animal personality and state-behaviour feedbacks: a review and guide for empiricists. Trends Ecol Evol. (2015) 30:50–60. doi: 10.1016/j.tree.2014.11.004
88. Beekman M, Jordan LA. Does the field of animal personality provide any new insights for behavioral ecology? Behav Ecol. (2017) 28:617–23. doi: 10.1093/beheco/arx022
89. Weiss A. Personality Traits: A view from the animal kingdom. J Pers. (2018) 86:12–22. doi: 10.1111/jopy.12310
90. Bombail V, Barret B, Raynaud A, Jerôme N, Saint-Albin A, Ridder C, et al. In search of stress odours across species: Behavioural responses of rats to faeces from chickens and rats subjected to various types of stressful events. Appl Anim Behav Sci. (2017) 205:216–26. doi: 10.1016/j.applanim.2017.10.013
91. Bombail V. Perception and emotions: On the relationships between stress and olfaction. Appl Anim Behav Sci. (2019) 212:98–108. doi: 10.1016/j.applanim.2018.12.013
Keywords: animal emotions, animal welfare, sensors, affective states, emotion modeling, animal-based measures, artificial intelligence
Citation: Neethirajan S (2021) The Use of Artificial Intelligence in Assessing Affective States in Livestock. Front. Vet. Sci. 8:715261. doi: 10.3389/fvets.2021.715261
Received: 26 May 2021; Accepted: 09 July 2021;
Published: 02 August 2021.
Edited by:
Antonio Lanatà, University of Florence, ItalyReviewed by:
Dana L. M. Campbell, Commonwealth Scientific and Industrial Research Organisation (CSIRO), AustraliaMarek Spinka, Czech University of Life Sciences Prague, Czechia
Copyright © 2021 Neethirajan. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Suresh Neethirajan, suresh.neethirajan@wur.nl