Thermal Imaging and Biometrical Thermography of Humpback Whales

Determining species distributions through time and space remains a primary challenge in cetacean science and conservation. For example, many whales migrate thousands of kilometers every year between remote seasonal habitats along migratory corridors that cross major shipping lanes and intensively harvested fisheries creating a dynamic spatial and temporal context that conservation decisions must take into account. Technological advances enabling automated whale detection have the potential to dramatically improve our knowledge of when and where whales are located, presenting opportunities to help minimize adverse human-whale interactions. Using thermographic data we show that near-horizontal (i.e. high zenith angle) infrared images of humpback whale (Megaptera novaeangliae) blows, dorsal fins, flukes and rostrums record similar magnitude brightness temperature anomalies relative to the adjacent ocean surface. Our results demonstrate that these anomalies are similar in both low latitude and high latitude environments despite a ~16°C difference in ocean surface temperature between study areas. We show that these similarities occur in both environments due to emissivity effects associated with oblique target imaging, rather than differences in cetacean thermoregulation. The consistent and reproducible brightness temperature anomalies we report provide important quantitative constraints that will help facilitate the development of transient temperature anomaly detection algorithms in diverse marine environments. Thermographic videography coupled with laser range finding further enables calculation of whale blow velocity, demonstrating that biometrical measurements are possible for near-horizontal datasets that otherwise suffer from emissivity effects. The thermographic research we present creates a platform for the delivery of 3 important contributions to cetacean conservation: 1) non-invasive species-level identifications based on whale blow shapes and velocities recorded by infrared videography; 2) reduced ship-strike rates through automated thermographic cetacean detection systems deployed in high traffic areas; 3) monitoring the spatial and temporal distributions of endangered animals in remote habitats.

Determining species' distributions through time and space remains a primary challenge in cetacean science and conservation. For example, many whales migrate thousands of kilometers every year between remote seasonal habitats along migratory corridors that cross major shipping lanes and intensively harvested fisheries, creating a dynamic spatial and temporal context that conservation decisions must take into account. Technological advances enabling automated whale detection have the potential to dramatically improve our knowledge of when and where whales are located, presenting opportunities to help minimize adverse human-whale interactions. Using thermographic data we show that near-horizontal (i.e., high zenith angle) infrared images of humpback whale (Megaptera novaeangliae) blows, dorsal fins, flukes and rostrums record similar magnitude brightness temperature anomalies relative to the adjacent ocean surface. Our results demonstrate that these anomalies are similar in both low latitude and high latitude environments despite a ∼16 • C difference in ocean surface temperature between study areas. We show that these similarities occur in both environments due to emissivity effects associated with oblique target imaging, rather than differences in cetacean thermoregulation. The consistent and reproducible brightness temperature anomalies we report provide important quantitative constraints that will help facilitate the development of transient temperature anomaly detection algorithms in diverse marine environments. Thermographic videography coupled with laser range finding further enables calculation of whale blow velocity, demonstrating that biometrical measurements are possible for near-horizontal datasets that otherwise suffer from emissivity effects. The thermographic research we present creates a platform for the delivery of three important contributions to cetacean conservation: (1) non-invasive species-level identifications based on whale blow shapes and velocities recorded by infrared videography; (2) reduced ship-strike rates through automated thermographic cetacean detection systems deployed in high traffic areas; (3) monitoring the spatial and temporal distributions of endangered animals in remote habitats.
Keywords: humpback whale, infrared, biometrical thermography, emissivity, blow velocity INTRODUCTION Since the International Whaling Commission (IWC) banned commercial whaling in 1986, many baleen whale species have shown signs of recovery (Thomas et al., 2016). However, all species listed as Least Concern under the International Union for Conservation of Nature and Natural Resources (IUCN) Red List of Threatened Species framework also include threatened subpopulations classified as Vulnerable, Endangered, or Critically Endangered (Thomas et al., 2016). The conservation status, recovery and health of whale populations is very much site and context specific: modern human threats to whales, including ship strikes and entanglement in fishing gear, are not evenly distributed with respect to both space and time (Thomas et al., 2016).
Despite the spatially and temporally dynamic challenges associated with cetacean conservation and protection in the postwhaling era, progress has been made. For example, revisions to shipping lane positions, vessel traffic management plans and mandatory maximum vessel speeds along the eastern coast of North America correlate with significant reductions in North Atlantic right whale (Eubalaena glacialis) deaths due to ship strikes (Laist et al., 2014;Thomas et al., 2016). However, the risks associated with ship strikes remain high elsewhere. Necropsies performed on stranded whales demonstrate that at least one humpback whale (Megaptera novaeangliae), one fin whale (Balaenoptera physalus), and two blue whales (Balaenoptera musculus) are killed by ship strikes off the California coast every year (Redfern et al., 2013). Similar analyses suggest that, on average, one Bryde's whale (Balaenoptera edeni) is killed every year by ship strikes in New Zealand's Hauraki Gulf (Constantine et al., 2015). True cetacean mortality rates due to human activities at sea are almost certainly higher (Kraus, 1990), however, and the annual loss of even a single individual can be significant for smaller populations of long-lived species with low recruitment rates (Laist et al., 2001).
In an effort to reduce the risks ships pose to large whales, the IWC has developed a 3-year (2017-2020) Strategic Plan that seeks to increase the development and use of whale avoidance technologies (Cates et al., 2017). Acoustic and infrared automated cetacean detection systems are attractive and emerging tools for enhanced cetacean conservation (Zitterbart et al., 2013;Nowacek et al., 2016). The ability to detect whale blows, as far away as 5 km using around-the-clock 360 • infrared scanners outfitted with rigorous detection algorithms (Zitterbart et al., 2013), will benefit many, including marine mammal observers onboard large vessels and land-based scientists studying whale movement behavior (e.g., Perryman et al., 1999) and human-whale interactions along rapidly changing coastlines (e.g., Graber, 2011). Infrared thermography can also facilitate the non-invasive collection and monitoring of fundamental biometrical information, including thermal physiology, injury diagnoses and population surveys (McCafferty, 2007).
Infrared cetacean detection systems also create opportunities for conservation biologists and cetacean ecologists to document the spatial and temporal distribution of animals utilizing remote or inaccessible environments. For example, the Oceania subpopulation of humpback whales, the only migratory humpback whales in danger of going extinct (Childerhouse et al., 2008), seasonally inhabit ∼10 million km 2 of the tropical South Pacific Ocean. Yet, only a handful of scientists, spread across an area of ocean the size of China, actively study these whales. Automated detection systems have the potential to dramatically improve our knowledge of when and where these endangered whales are utilizing highly understudied breeding/calving ground habitats.
However, thermal imaging also has several important limitations. Infrared imaging systems are not inexpensive, particularly so for current high sensitivity models with cryogenically cooled detectors or large focal lengths capable of long-range applications. Infrared detectors also require a direct line of site to the target, yet they can also lose functionality through interaction with sea-spray. The data streams generated by infrared imaging systems are large, creating challenges with data handling, analysis and signal processing. Thermal cameras are also highly inaccurate when imaging scenes from nearhorizontal positions due to emissivity effects (Masuda et al., 1988;Cuyler et al., 1992;see Nomenclature).
The effects of emissivity on the brightness temperatures recorded by a thermal imaging device are extremely relevant to cetacean thermography. For example, as a whale exhales, its breath pushes sea water present in the near-surface water column, or nasal depression, or both, into the overlying atmosphere. From observation points at or near sea-level, this spouting of water droplets immediately and drastically changes the angle at which the whale's blow is being measured by the thermal camera. For example, a 2 m high whale blow will be measured perpendicularly (i.e., measured at a 0 • zenith angle) from an observation point located 100 m distant and 2 m above sea level. In contrast, the adjacent flat ocean's surface will be measured sub-horizontally at an 89 • zenith angle. Similar to blows, emergent dorsal fins, flukes or rostrums will also be measured at a relatively low zenith angles in relation to the adjacent ocean's surface. These rapid changes in the angle at which the object is being imaged will have large effects on the surface brightness temperatures estimated by the thermal imaging device due to the effect zenith angle of observed radiation has on sea water emissivity (Masuda et al., 1988).
The research we present was driven by three primary objectives, all aligned to the IWC's strategic goal of developing large whale avoidance technologies. We sought to: (1) quantify infrared image brightness temperature and brightness temperature anomaly (BTA) values for humpback whale blows, dorsal fins, flukes, and rostrums in both tropical breeding/calving ground and sub-polar feeding ground habitats; (2) calculate humpback whale blow height and blow velocity through coupling of infrared videography with laser-range finding; (3) evaluate the effects of emissivity on thermal imaging data collected from high zenith angle (i.e., oblique to target) positions. Achievement of these objectives creates a platform from which a variety of cetacean conservation tools can be further developed and delivered.

METHODS
Thermal images of humpback whale surfacing features were collected using a Forward Looking Infrared camera (FLIR A615, FLIR Systems, Inc.,) and analyzed using FLIR Tools+ software (FLIR Systems, Inc.,). The FLIR A615 we used had a focal length of 24.6 mm, 25 • × 19 • field of view, F-number of 1.0, infrared resolution of 480 × 640 pixels and a detector pixel pitch of 0.017 mm pixel −1 . The camera's detector comprised an uncooled Vanadium Oxide (VoX) long-wavelength (i.e., 7.5-14 µm) microbolometer (see Nomenclature) with a thermal sensitivity of <0.05 • C. Infrared images were captured every 0.04 s (i.e., 25 Hz) but frame rates as high as 200 Hz can be achieved with the A615's high-speed windowing option. The A615 was powered by a small 12-volt battery externally strapped to the camera's casing. The A615 was also connected to a FZ-G1 ToughPad tablet computer (Panasonic Corporation) via a high-speed USB cable. A GoPro Hero4 camera (GoPro, Inc.,) was affixed to the top of the A615 for contemporaneous visible wavelength image collection. This study was carried out in accordance with the recommendations of the Cook Islands Government. The protocol was approved by the Office of the Prime Minister, Cook Islands Government.
In Rarotonga, infrared and visible wavelength images were collected either ∼2 m above the ocean surface while onboard a Cook Islands Whale Research vessel, or from shore-based positions ∼5-10 m above sea level on the island's northwest coast (Figure 1). In Sitka Sound, all images were recorded ∼4 m above the ocean surface while onboard a commercial whale watching cruise arranged by the Sitka Sound Science Center as part of the annual Sitka Whale Fest (e.g., Figures 1D,E). Despite these variable imaging heights, our entire dataset was collected at >85 • zenith angles (i.e., <5 • off horizontal) due to the range in distances at which whales were imaged. A Nikon Forestry Pro laser rangefinder was used to determine whale distances whenever possible. All measurements were made during Beaufort wind force scale numbers 2-4 and similarly ranked World Meteorological Organization (WMO) Sea State codes.
Brightness temperatures were extracted from individual thermal images using the line measurement tool available in FLIR Tools+. Two lines for temperature data extraction were drawn across each image: the first line was drawn vertically through the background scene immediately adjacent to the targeted whale feature (i.e., blow, dorsal fin, fluke, rostrum, Figure 1), and the second line was drawn vertically such that it passed through the maximum brightness temperature included within the targeted whale feature. Thermal benchmarks included within each image, such as the steep thermal gradient across the ocean-atmosphere boundary, were used to align the pixels included in each line's thermal profile (Figure 2). Once aligned, the brightness temperatures recorded by each line were subtracted from each other in order to calculate BTA-values at the individual pixel scale for each whale feature analyzed (Figure 2).
Because the A615's pixel pitch and focal length were known, independent measurement of whale distances by laser rangefinding allowed us to estimate blow height from thermal image pixel measurements by combining the optical lens equation, with the magnification equation, and the camera's pixel pitch, .
( 3) Equation (1) can be rearranged to, Equation (2) can be arranged to, and equation (3) can be rearranged to, Substituting equations (4) and (6) into equation (5) gives, Blow heights were estimated using Equation (8) every 0.04 s following blow initiation. Image pixel heights were measured using FLIR Tools+ and blow distances were measured by laser range-finding as described above.

RESULTS
In total, we determined BTA profiles for 174 humpback whale blows, 20 dorsal fins, 9 flukes, and 20 rostrums. An equivalent number of whale features were analyzed from each of the two study areas, with the exception of flukes, for which 6 were imaged in Alaska and only 3 were imaged in Rarotonga. Of the 87 blows analyzed in each study area, 32 Rarotonga blows and 16 Alaska blows were imaged at distances <150 m. Of these, only 10 blows from each study area were recorded in the 100-150 m range. Average BTA profiles demonstrate that humpback whale blows, dorsal fins, flukes and rostrums appear as thermal anomalies of similar magnitude relative to adjacent ocean water (Figure 3). For example, 100-150 m distant blows in Rarotonga and Alaska appear as 20-30 pixel-wide thermal anomalies that are ∼3 • C warmer than the adjacent ocean ( Figure 3A). Similarly, dorsal fins and flukes in both areas exhibited maximum BTA values ca. 3-4 • C (Figures 3B,C), whereas rostrums from both populations were ∼2-3 • C warmer than the adjacent ocean ( Figure 3D). Ocean water temperatures were measured by perpendicular thermography and satellite observations in both study areas. These measurements indicate surface ocean water temperature was ∼24 • C in Rarotonga, and ∼8 • C in Sitka Sound, Alaska, at the time thermal images were recorded.
The shapes of the average dorsal fin, fluke and rostrum BTA profiles differ because these features were recorded across a large range of distances in each study area. Because the Rarotonga whales were generally imaged at closer ranges, the dorsal fin, fluke and rostrum BTA profiles are spread across a larger number of image pixels than the Sitka BTA profiles (Figures 3B,D). In other words, the Rarotonga whale features fill a larger portion of the 640 × 480 pixel thermal images because these images were recorded at closer distances. Despite these distance-related differences in BTA profile shape between the study areas, the maximum BTA values for humpback whale blows, dorsal fins, flukes and rostrums (indicated by arrows in Figure 3D) we recorded are not significantly different (p >> 0.05, two-tailed t-test, Figure 3).
Laser range-finding enabled quantification of the relationship between the pixel-length of individual blows and blow distance for the FLIR A615. As expected, blow pixel-lengths are larger for images recorded at closer range, and blow pixel-length decreases FIGURE 2 | FLIR Tools+ thermographic image of Rarotonga blow #63 (i.e., RB63) recorded at a distance of 87 m using a FLIR A615 infrared camera (A), including two brightness temperature extraction control lines (Li1 and Li2 in A) drawn adjacent to the targeted blow for brightness temperature extraction along the third line (Li3 in A). (B) displays the raw brightness temperature profiles recorded by the A615 camera for all three lines shown in (A), and (C) displays the brightness temperature anomaly of the blow (Li3) relative to background brightness temperatures (Li1). Inset panel in (A) shows a portion of the same scene as recorded by a visible wavelength GoPro Hero4 camera attached to the top of the A615 thermal camera. Prominent features of both the infrared and visible wavelength images are labeled for reference. with blow distance according to an inverse power relationship (Figure 4). Although blows imaged at <200 m range were easily recognizable with the A615 (Figures 4A,B,D), a blow imaged at ∼400 m range appeared as an 8 pixel tall ∼0.4 • C BTA ( Figure 4E). Higher sensitivity cooled detector thermal imaging devices and/or devices with longer focal lengths would no doubt extend the range at which whale blows might be detectable (e.g., Zitterbart et al., 2013). However, these larger systems are FIGURE 3 | Thermographic profiles of emergent humpback whale blows and body parts in both tropical (Rarotonga, Cook Islands) and sub-polar (Sitka Sound, Alaska, U.S.A.) waters. Each plot presents the average brightness temperature difference relative to background ocean water brightness temperatures for blows (A), dorsal fins (B), flukes (C), nostrils and rostrums (D). Only rostrums were imaged in Alaska, whereas both nostrils and adjacent rostrums were imaged in Rarotonga. Ten individual profiles from each of the study areas were used to construct the average brightness temperature anomaly (BTA) profiles shown, with the exception of flukes, where only 6 fluke profiles were recorded in Alaska and 3 fluke profiles in Rarotonga. Shaded regions correspond with ± 1 SD brightness temperature variation. Individual profiles were aligned such that the image pixel with the largest brightness temperature difference relative to seawater was assigned pixel number zero. Negative pixel numbers correspond with pixels that are skyward of the maximum brightness temperature difference pixel. Positive pixel numbers correspond with pixels that are seaward of the maximum brightness temperature difference pixel. The parabolic shape of each average profile reflects the fact that the individual datasets used to determine the average profiles shown were imaged at different distances with correspondingly different image pixel widths/lengths. For example, because the Rarotonga rostrums were imaged at closer range than the Sitka rostrums, the Rarotonga rostrums span a much larger number of pixels and include positive thermal anomalies across the nostrils that were not captured in any of the Sitka images (D).
currently much more expensive and less maneuverable than the FLIR A615 we used here.
Regardless of the device used or its imaging range, whale blow heights will also vary in response to a number of uncontrollable factors, including: wind shear, the volume of sea water in the nasal depression at exhalation, and the whale's position relative to the ocean surface at which exhalation is initiated. In an effort to partially overcome these complicating factors, we calculated blow heights 0.4 s after blow initiation, the minimum observed period for a blow to achieve its maximum height, for 32 humpback whale blows across an 18-140 m range in distances (mean = 71 ± 38 m, ± SD, Figure 4). The pixel height (range = 24-230 pixels, mean = 63 pixels ± 46 pixels, ± SD, Figure 4) of each imaged and laser ranged blow was measured using FLIR Tools+. Estimated blow heights at 0.4 s ranged between 1.0 and 3.3 m (mean = 2.2 ± 0.5 m, ± SD, n = 32). In addition to wind, water volume, and whale position, blow heights are also likely to vary with the volume of air being expelled in a specific exhalation. Although untested, focal follows incorporating thermal imaging techniques have the potential to reveal the breathing behaviors of individual whales of different size, maturity, sex and physiological condition.
Utilization of the 25 frames per second videography option enabled us to also estimate humpback whale blow velocity ( Figure 5). All blows analyzed reached maximum blow height in <1.2 s and the maximum blow height measured was 4.7 m at 0.8 s following blow initiation equating to a 21 km h −1 velocity for this blow (Figures 5G,J,K). Notably, some blows were unambiguously initiated while the nostrils/blowholes were still submerged. Blows of this type exhibited a relatively slow initial acceleration (e.g., Figures 5G-I) as the exhaled air pushed into the overlying water column. Individual blows exhibited maximum blow velocities that ranged between 40 and 55 km h −1 (mean = 13-23 km h −1 ± 12-18 km h −1 , ±SD). Maximum blow heights ranged between 2.7 and 4.7 m and occurred 0.76-1.16 s following blow initiation. At 0.4 s following exhalation initiation, the humpback whale blows we recorded were 1.4-3.3 m tall. It is important to acknowledge that these estimates are derived from the thermal anomalies associated with water droplets that are blasted out of the ocean's surface or nasal depression by exhaled air. Thus, the velocities we calculated must be considered minimum estimates of the true gaseous exhalation velocities achieved by humpback whales.
Our results demonstrate that humpback whale blows, dorsal fins, flukes and rostrums present as similar magnitude brightness temperature anomalies (BTA) in both tropical (Rarotonga, Cook Islands) and sub-polar (Sitka Sound, Alaska, U.S.A.) environments despite an ∼16 • C difference in ocean surface temperature between the two study areas. This occurs due to emissivity effects associated with the oblique nearhorizontal imaging angles used in the current study. Thus, absolute temperatures determined from oblique (i.e., subparallel to target) measurement angles do not represent accurate quantifications of whale blow or skin temperatures. Our results also demonstrate how to calculate blow heights and blow velocities by combining target BTA pixel size with target distance as measured by a laser range finder. Although blow acceleration varied both within and between individual blows, our results indicate that humpback whale blows have average instantaneous velocities of ∼4.6 m s −1 .

DISCUSSION
The infrared radiation emitted by a surface is a function of both the surface's temperature and its spectral emissivity (see Nomenclature). Thermal imaging systems estimate surface temperatures by assigning emissivity values to the imaged scene. However, sea surface emissivity, the ratio of the energy radiated from the ocean's surface relative to a blackbody, further depends on the ocean's surface roughness, refractive index, and the zenith angle from which the surface is being observed (Masuda et al., 1988). Thus, quantitative analyses and accurate interpretations of thermographic datasets collected at sea depend on a large number of variables.
Of these variables, the angle from which the surface is being observed has the largest effect on emissivity and, as a consequence, thermographic temperature estimates (Masuda et al., 1988). For example, the emissivity of perfectly planar sea water at a 0 • zenith angle (i.e., perpendicular to the sea surface) is ∼0.98 (Masuda et al., 1988). At a 60 • zenith angle this same surface will have an emissivity of ∼0.92 and at 85 • (i.e., 5 • above the horizontal) the emissivity drops to ∼0.36 (Masuda et al., 1988). Using human targets included in our thermographic image dataset, we found that a decrease in surface emissivity of 0.98-0.36 resulted in a 12.2 • C increase in the human skin surface temperature reported by the camera. Similar tests on 25 • C ocean water revealed that a similar magnitude change in emissivity resulted in a 3.5 • C change in sea surface temperature at ∼100 m distance. As suggested by Cuyler et al. (1992), our findings confirm it is inappropriate to assume relatively high emissivity values (i.e., >0.95) in thermographic cetacean research when imaging is performed at high zenith angles.
Thus, the data we report suffers from extreme emissivity effects due to the fact that our thermal images were collected at sub-horizontal observation angles (i.e., zenith angles of ∼85 • -89 • ). However, the A615 infrared camera we used includes a high sensitivity microbolometer (<0.05 • C); thus, the brightness temperature measurements we report can be considered precise but not accurate. Although the loss of thermographic accuracy due to emissivity effects associated with oblique-angle imaging is problematic for biometrical estimates of cetacean thermoregulation, it is a benefit to cetacean detection.
Brightness temperature anomalies of ∼2-4 • C, like those we report for humpback whale blows, dorsal fins, flukes and rostrums (Figure 3), are the consequence of rapid changes in emissivity as the whale feature emerges from the ocean's surface and immediately changes the observation point zenith angle. However, the higher BTA values we report for humpback whale nostrils (ca. 4.5 • C, Figure 3D) likely reflect a more accurate approximation of humpback whale skin temperatures due to the closer range at which nostrils were imaged (i.e., at lower zenith angle) and the observed ∼2 • C difference between nostril/blowhole temperatures and adjacent (wet) rostrums ( Figure 2D). The potential utility of thermographic imaging of cetacean nostrils/blowholes for biometrical research purposes should be more deeply explored using aerial drones mounted with high resolution and high frame-rate thermal imaging systems.
One of the primary challenges in cetacean ecology and conservation is determining when and where whales are located. Although our results do not include accurate determinations of whale surface temperatures, they conclusively demonstrate that whale blows and emergent body parts appear as similar magnitude thermal anomalies, ca. 2-4 • C, relative to surface waters in both tropical and sub-polar environments at distance ranging between 100 and 150 m. These thermal anomalies are largely due to emissivity effects associated with thermographic imaging from sub-horizontal positions. Thus, our findings represent an important quantification of the magnitude of the thermal signal from which thermographic cetacean detection algorithms can be developed and refined.
Quantitative constraints on the magnitude, size and duration of whale-derived thermal anomalies can also be used to help restrict the number of false positives and false negatives produced by automated cetacean detection systems that use transient thermal contrast algorithms based on average brightness temperatures (e.g., Zitterbart et al., 2013). Improving automated detection systems in this way should assist applications in windy conditions or large swells, when ocean surface roughness has the potential to produce thermal anomalies of similar magnitude as whale blows due to emissivity effects (e.g., Figure 4E). Differentiating cetacean induced anomalies from non-cetacean induced anomalies will also benefit from quantifications of thermal anomaly shapes and their evolution through time. For example, our results demonstrate that the water spouts produced by humpback whale exhalations move at ∼4.6 m s −1 and accelerate at ∼100-300 m s −2 . Such biometrical measurements not only provide additional quantifications for the development of automated cetacean detection systems, but also create a platform for species-level identifications using measurable differences in blow geometry and velocity.

AUTHOR CONTRIBUTIONS
TH and PZ-R conceived of the study. All authors contributed to the fieldwork and infrared imaging in Rarotonga, Cook Islands. TH performed the fieldwork and infrared imaging in Sitka Sound, Alaska, U.S.A. TH and AO performed all of the thermographic image processing and analysis. TH and AO wrote the initial manuscript and all authors contributed to the revision of the manuscript.