Fast Timing Techniques in FLIM Applications

Fluorescence lifetime imaging (FLIM) is increasingly used in many scientific disciplines, including biological and medical research, materials science and chemistry. The fluorescence label is not only used to indicate its location, but also to probe its immediate environment, via its fluorescence lifetime. This allows FLIM to monitor and image the cellular microenvironment including the interaction between proteins in their natural environment. It does so with high specificity and sensitivity in a non-destructive and minimally invasive manner, providing both structural and functional information. Time-Correlated Single Photon Counting (TCSPC) is a popular, widely used, robust and mature method to perform FLIM measurements. It is a sensitive, accurate and precise method of measuring photon arrival times after an excitation pulse, with the arrival times not affected by photobleaching, excitation or fluorescence intensity fluctuations. It has a very large dynamic range, and only needs a low illumination intensity. Different methods have been developed to advance fast and accurate timing of photon arrival. In this review a brief history of the development of these methods is given, and their merits are discussed in the context of their applications in FLIM.


Early Timing of Light Signals
The quest for precise timing of light signals can be traced back to 1638, when Galileo Galilei (1564-1642), performed an experiment using two observers with lanterns and manual shutters stationed on two well-separated hilltops. Although he suspected that light did not travel instantaneously, this kind of terrestrial approach was far too slow to observe the speed of light experimentally. Astronomical measurements provided a way forward: In 1675, the Danish astronomer Ole Rømer (1644-1710) observed the eclipses of the innermost moon of Jupiter, Io, at different times of the year and proposed a finite speed of light to account for timing differences of the moon emerging from the shadow of Jupiter, as seen from earth. In addition, in another astronomical approach around 50 years later, in 1728, the British astronomer James Bradley (1693-1762) calculated the speed of light from stellar aberration [1].
First terrestrial measurements of the speed of light were performed by Hippolyte Fizeau (1819-1896) in 1848-49 by reflecting an intense beam of light from a mirror 8 km away through a toothed wheel, and in 1850-62 Léon Foucault (1819-1868) improved the experiment by replacing the cog wheel with a rotating mirror, and eventually measured the speed of light within 0.6% of today's accepted value, and Maxwell's calculations [2]. At King's College London, Charles Wheatstone (1802-1875) also experimented with rotating mirrors to measure the duration of electric sparks [3].

The Photon
After the experimental determination of the speed of light, Maxwell's prediction of electromagnetic waves [2] and their experimental detection by Heinrich Hertz in the 1880s, the nature of light came under increased scrutiny. Max Planck's attempt to explain the blackbody spectrum in 1900 ushered in the idea of quantization of energy. Planck considered hypothetical material oscillators that can only emit and absorb electromagnetic radiation in discrete, quantized form and not in continuously varying quantities, especially not in arbitrarily small amounts. Although he did not explicitly propose that light is quantized, he proposed that the energy ǫ of these oscillators is proportional to their oscillation frequency ν ǫ = hν (1) where h is now known as Planck's constant. In 1905, Albert Einstein [4] introduced the smallest unit of energy of electromagnetic radiation, Lichtquant (quantum of light), and explained the photoelectric effect using this concept. The term photon was conceived by the chemist Lewis in 1926 [5], and the existence of light quanta as well as the term photon gained wider acceptance as quantum theory was developed in the 1920s. In particle physics, the photon is an integer spin particle, a boson, with spin 1. A photon is a "quantized field, " it always travels at the speed of light and has a momentum, but no mass, and no charge [6]. It does not decay, once it is emitted, its frequency stays fixed 1 .

Experimental Detection of Photons and Photon Arrival Timing
Experimental detection of single photons requires that the signal from a single photon is converted into an electronic signal. The photoelectric effect, first observed by Heinrich Hertz in 1887 [7], provides the means to do this. While the electronic signal of one photoelectron is too small to be detected, the secondary emission principle, discovered by Austin and Starke [8], allows electron multiplication and hence gain to be introduced. Thus, the photoelectron signal, created by a single photon, can be amplified and measured.
The capability to detect single photons means that it is also possible to time their arrival, similar to timing the arrival of particles. Indeed, the origin of time-correlated single photon counting (TCSPC) lies in particle physics. In 1929, Walther Bothe and Werner Kohlhörster measured the coincidence of penetrating charged particles in cosmic rays [9] leading to the first practical electronic coincidence circuit [10] which became a precursor of the AND logic circuits of electronic computers. By the addition of a delay, this coincidence method evolved to measure delayed coincidence, providing the means for timeresolved measurements and the invention of the "time circuit, " nowadays known as the Time-to-Amplitude Converter (TAC) [11]. It became a popular method to measure short radioactive decay times, with time resolution improved to 80 ps in the 1950s [12]. Scintillation decay times, excited by gamma rays, were also reported in the 1950s, using a delayed coincidence method [13].
In 1961, Lowell Bollinger and George Thomas generalized the scintillation measurements to include any type of radiation [14], and flashlamps with optical pulse widths of around 2 ns became available in the 1960s, enabling TCSPC: This is essentially a delayed coincidence method, whereby the arrival time of a single photon is measured relative to an excitation pulse, and this can be done with a few picosecond precision [15]. The accumulation of the arrival time of many single photons then represents the intensity decay of the sample, as long as no photons are lost due to pile-up, and the linearity between intensity and collected photons holds. See Figure 1 for a timeline of the development of single photon counting and timing technology.

Fast Timing in Fluorescence
In fluorescence measurements, the sample is excited with a short pulse of light and the decay of fluorescence is measured. This approach can be traced back to the phosphoroscope developed in 1857 by Alexandre-Edmond Becquerel (1820-1891), based on a similar principle to Foucault's speed of light measurement experiments. Becquerel used two rotating disks with a series of holes, and the sample in between, to measure photoluminescence lifetimes. In his apparatus, the sample (e.g., a phosphorescent crystal) was excited by light coming in through one hole, and viewed by the phosphorescent light coming out of the other hole. By varying the speed of rotation, Becquerel was able to measure the short microsecond time interval during which the phosphorescent light is emitted.
In modern time-correlated single photon counting experiments, the photons emitted by the sample are timed one by one and used to build a histogram of the fluorescence decay over time, see Figure 2A. In the simplest case, the fluorescence decay is a monoexponential function where I 0 represents the fluorescence intensity at t = 0 [16]. The fluorescence decay time τ is the average time the fluorophore remains in the excited state and is defined by where k r is the radiative rate constant which is largely determined by the extinction coefficient, the fluorescence spectrum and the refractive index of the fluorophore's environment [17]. The nonradiative rate constant k nr typically depends on the environment, and can change with interaction, pH, viscosity, and other parameters [16]. Fluorescence decay measurements in fluorescence spectroscopy or fluorescence microscopy can therefore be used to gain information about the immediate environment of the fluorophore, as well as to probe the proximity of other fluorophores via Förster Resonance Energy Transfer (FRET) [18][19][20]. Moreover, the fluorescence decay is typically independent of the fluorophore concentration at concentrations low enough to avoid interaction or aggregation.
The first reports that use TCSPC in the measurement of fluorescence decays appear in the early 1970s [21][22][23][24], and TCSPC was soon widely used for time-resolved spectroscopy, and in particular the measurement of fluorescence lifetimes in solution [25].
Photon counting techniques are used in many different fields of science and technology, from DNA sequencing [26] to quantum cryptography and interplanetary communications [27], and photon timing applications include light detection and ranging (LIDAR) [28,29], photon correlation techniques [30], and optical tomography [31]. Single photon detection techniques and applications have been reviewed recently by Buller and Collins [32], Hadfield [33] and Eisaman et al. [34]. Incidentally, the same technology is not only used for photon arrival timing, but also particle arrival timing for particles such as neutrons, where timing can indicate neutron energy [35,36], and electrons and ions, e.g., in ion velocity mapping [37][38][39], so there is a growing need for this technology, [40] with applications beyond timing the arrival of photons.

Fluorescence Lifetime Imaging (FLIM)
The development of laser scanning confocal microscopes [41] enabled TCSPC-based fluorescence lifetime imaging (FLIM), as shown in Figure 2B. This method was developed in the late 1980s [42] and early 1990s [43][44][45] before being more widely applied in the late 1990s [46]. Besides scanning with a point-detector to form an image of the sample, microchannel plate (MCP)-based detectors and more recently single photon avalanche diode (SPAD) arrays can be used for wide-field TCSPC FLIM [47]. The development of both single point and widefield single photon detection methods has continued over the decades, and while the old technologies continue to be used and improved, there have also been new developments that show great promise, including SPAD arrays and detectors based on superconducting detector technology. See Figure 1 for a timeline of single photon technology.
The specific requirements for the detectors and the timing electronics depend on the field of application. This review gives an overview of the technology behind the different approaches to fast photon timing and recent developments with a focus on fluorescence decay measurements and particularly FLIM.

Principles Behind Photon Timing
Photon timing means that the arrival time of a photon at the detector is measured with respect to some reference, typically the excitation laser pulse. A trigger from the laser and the signal from the detector are fed into the timing circuit and their difference is calculated. The photons are then usually assigned a "time bin, " i.e., a slot whose width usually depends on the detector precision, the required measurement precision, and the total length of the decay to be measured, see Figure 3. The circuit is adjusted such that the entire decay fits into the available time window, typically 5 times the length of the fluorescence lifetime, so that it can decay into the background noise. The measurement is repeated for many photons, until the arrival time histogram-representing the fluorescence decay-contains a statistically significant number of photons for fluorescence decay analysis, and to extract the fluorescence lifetime [48][49][50][51].

Instrument Response
A photon timing detector is usually characterized by the full width at half maximum (FWHM) of its instrument response function (IRF), which is measured using a very short pulse of light. In an ideal case, the measured IRF would be a Dirac delta function in a single time bin, but due to experimental uncertainties it is always wider. The width of the instrumental response t 2 IRF is given by the sum of the squares of the individual time spread contributions [52] Frontiers in Physics | www.frontiersin.org  where t optical is the optical pulse width of the exciting light, t tts is the transit time spread of the detector, and t jitter is the jitter of the detection and timing electronics. Optical pulse widths can be as short as femtoseconds, e.g., for two-photon excitation, and the jitter of the electronics can be around 1 ps for the fastest timing systems.

The Poisson Statistics of Photon Counting
One key advantage of photon counting is the well-defined theoretical framework of Poisson statistics, which govern the photon counting process. The photon events are independent, i.e., in ensemble measurements the detection of one photon event does not change the probability of detecting another one. (For single photon work, this idea is modified, and in Hanbury Brown and Twiss-type experiments, this is exploited to gain information about coincident events.) The variance is the mean of the distribution, and the experimental uncertainty in the form of the standard deviation is given by the square root of the number of counts. This also means that the more photons are collected, the smaller the experimental uncertainty, and in practice the dynamic range is only limited by the detector and electronics non-linearities. Poisson statistics also predict how often multiple photon events are detected as a function of the mean count rate. Importantly, the probability that more than one photon occurs after an excitation event is never zero, and there are excitation cycles where this happens. If only one photon can be detected after an excitation event, then the first photon arriving is timed, but subsequent photons are lost. This is known as pile-up which distorts the fluorescence decay toward shorter times [15]. In practice, in the older spectroscopy literature it is recommended to keep the stop-to-start ratio (ratio of fluorescence photon count rate to repetition rate of the excitation laser) to around 1%, for fluorescence decays with typically 10,000 counts in the peak [15].
Again, it is worth emphasizing that pile-up always occurs, the probability to detect several photons after one excitation pulse is never zero-the question is whether the effect emerges from the Poisson noise (square root of the number of counts). For FLIM, with typically fewer counts in the fluorescence decays in each pixel than in spectroscopy measurements, the 1% stop-to-start ratio can be somewhat relaxed, and a 10% stop-to-start ratio only introduces a 2.5% fluorescence lifetime error [49]. If all photons that are emitted after an excitation event are detected, then this is not a problem [53].

SINGLE PHOTON DETECTORS
The detection of a single photon requires that (1) the photon is converted into an electronic signal, and (2) that this signal is larger than the noise, so it can easily be measured-in practice, it needs to be amplified. Different detectors have different characteristics regarding transit time spread, detection efficiency, noise, ease-of-use and durability. Detectors used in TCSPC/FLIM applications are discussed in this section, see Table 1 for a summary.

Photomultiplier Tube (PMT)
The first working PMTs were produced in the former Soviet Union in the mid-1930s [54][55][56] in the race to develop electronic television, and were soon shown to be single photon sensitive [57]. A PMT is a vacuum device that consists of a photocathode, up to a dozen discrete dynodes, each at a higher potential than the previous one, and an anode, see Figure 4 [58][59][60][61]. A photon hitting the photocathode can liberate a photoelectron, which in turn releases secondary electrons when it hits the first dynode, attracted to move toward it by the positive potential. The electric signal is amplified in the same way at each dynode, and the overall gain of a PMT can reach values of 10 6 to 10 8 which yields an easily measurable current at the anode, or a pulse. If two photons are detected in quick succession, the output pulses can merge and increase the pulse height. This effect has been exploited to design a so-called pulse pile-up inspector that rejects pulses over a certain pulse height and thus reject multiphoton events which can bias a fluorescence decay [21,62]. The reproducibility of measurements is affected by the electron transit time, anode pulse rise time, and electron transit time spread. The transit time spread determines the full width half maximum of the IRF. For example, according to its data sheet 2 , the Hamamatsu R14755U PMT's single photon pulse has a rise time of 400 ps, but a transit time spread of only 200 ps. Short rise times are advantageous for precise timing. Incidentally, the IRF is much smaller than the width of the single-photon output 2 https://www.hamamatsu.com/eu/en/product/type/R14755U-100/index.html pulse of the PMT, which is quoted as 680 ps. The spectral response characteristics are determined on the long wavelength side by the photocathode material and on the short wavelength side by the window material. PMTs require a vacuum and a high voltage for their operation, and are sensitive to strong magnetic fields.
Photon detection efficiency of PMTs depends strongly on the photocathode. The detection efficiency of conventional bialkali and multialkali cathodes reaches 20-25% between 400 and 500 nm, but falls off quickly toward longer wavelengths. GaAsP cathodes can reach up to 50% detection efficiency and work up to 700 nm. However, dark current also depends on the cathode material and can become a problem especially with materials that have increased sensitivity toward infrared wavelengths. Typical dead-times of PMTs are in the order of 10 ns, which is generally lower than the dead-time of the electronics used for photon timing and therefore usually not a concern for photon timing applications. They also usually have a large detector area compared to diodes-50 cm diameter for the Super-Kamiokande PMTs [63]-which makes them easy to align in confocal or multiphoton excitation fluorescence microscopes. Multianode PMTs have been produced, with 16 anodes, and have been employed in spectral FLIM [64,65].

Microchannel Plate (MCP)
The concept of a microchannel plate (MCP) with a continuous dynode was conceived in the 1930s [66] (coincidentally around the same time as PMTs), although the first working devices were not produced until the early 1960s [67][68][69]. Like PMTs, FIGURE 4 | Schematic of a photomultiplier tube. A photon impinging on the photocathode can liberate a photoelectron which is attracted to the first dynode, where further electrons are created by secondary emission. After around 10 dynodes, the gain can be well over 10 6 , producing an output pulse that can be timed by timing electronics [58][59][60][61].
the operating principle of MCPs is also based on the signal from the emission of a photoelectron from a photocathode and its amplification via secondary emission, but an MCP has continuous dynodes rather than discrete dynodes, as shown in Figure 5. This shortens the transit time, and hence also the transit time spread, which is a critical parameter for photon timing applications. MCPs have a regular array of tiny tubes (microchannels) where electrons are accelerated from one side of the plate to the other through a high voltage (typically a few kV). The microchannels are usually straight and round with ∼10 µm in diameter with a center-to-center distance of ∼15 µm [70,71]. They are arranged at an angle toward the normal (bias angle), so that ion feedback is minimized. Most MCP detectors consist of two (chevron configuration) or three (z-stack configuration) MCPs either pressed together or with a small gap between them, and the bias angles of the plates are rotated 180 • with respect to each other to prevent ion feedback. One MCP can have gain up to 10,000, but a 3-MCP intensifier can provide gain > 10 7 .
MCP detectors do not in principle have "dead-time" when the detector is not capable of detecting new events. However they can be affected by gain depletion where electrons are not delivered fast enough to an area where many events occur, leading to reduced electron gain and consequently reduced output signal amplitude. This can become an issue especially with camerabased read-out which can detect up to 100s of events per frame [47], but this is unlikely to become an issue if the MCP is used for timing photons one by one.
Inside MCPs the electron transit time is short, and consequently, the transit time spread is also short, 10s of ps. Indeed, MCPs are used as fast detectors for timing in TCSPC, and IRFs of <20 ps have been reported [48,51]. The output electron cloud can then be detected by several methods, depending on the application.
If no position read-out is required, the total current can be converted to a voltage via a resistor and read out as a pulse, as schematically indicated in Figure 5. For wide-field TCSPC, a position-sensitive read-out is necessary, and the electrons can be detected with either a position-sensitive anode, or converted to photons with a phosphor screen and detected with a camera FIGURE 5 | Schematic of a microchannel plate. A photon impinging on the photocathode can liberate a photoelectron, which is attracted into the first microchannel plate where secondary emission inside the microchannels produces more electrons. They can be converted into a timing pulse at the anode. [47]. Large area MCPs with dimensions of 20×20 cm 2 have been reported [72].
Although MCPs are a mature technology, they are still widely used and their development continues. For example, unlike PMTs, MCPs can provide spatial resolution, which is advantageous in low light level imaging. Photon counting imaging, where the amplified photon events were imaged with a camera and the image assembled photon by photon, was employed by astronomers due to the exquisite sensitivity this method offered at the time [73]. Indeed, the Hubble space telescope's faint object camera was based on photon counting imaging [74]. The x-ray multimirror satellite's optical monitor [75], the ultra-violet/optical telescope (UVOT) on the Swift gamma-ray observatory [76] and the Galaxy Evolution Explorer (GALEX) satellite [77] all employ MCP-based photon counting image intensifiers. Developments in the field are ongoing [78,79], and recent improvements using atomic layer deposition include robust substrates that able to withstand high processing temperatures, very low background rates, high stable gains, and low outgassing [80,81].

Avalanche Photodiodes (APD/SPAD)
In contrast to PMTs and MCPs that are vacuum devices operated at kV voltages, avalanche photodiodes (APDs) are semiconductor solid-state devices based on a p-n junction, for which no vacuum and kV voltages are required. A diode is an electronic component that allows current to flow through it in one direction (when biased in forward mode), but not the other (reverse mode), as schematically shown in Figure 6A. A photodiode has a depletion region which is largely free of mobile charge carriers, but when a photon is absorbed in it, an electron-hole pair is created, see Figure 6B. Applying an electric field (reverse bias) will allow the holes to go to the anode, and the electrons to the cathode-thus a flow of current is created by absorbing photons in the depletion region of a photodiode when biased in reverse mode.
By applying a high reverse bias voltage (typically 100-200 V in silicon), APDs show an internal current gain effect (around 100) due to impact ionization (avalanche effect). Gain can be improved by different doping and beveling techniques. A key feature of the electron-hole pair generation is that unlike in the photoelectric effect, this is a process for which no vacuum is required.
SPADs are APDs that are operated with a reverse voltage above a typical APD's breakdown voltage. This is also called Geiger mode, and it can achieve gains of 10 5 -10 6 . This mode is particularly useful for single-photon detection. Once a photon has been absorbed and an electron-hole pair created, an electron avalanche is initiated which reduces the reverse bias below the breakdown voltage and eventually stops the avalanche. If the SPAD is designed for passive quenching, a large resistor in series to the photodiode quenches the avalanche, whereas in active quenching, an electronic circuit is triggered by the avalanche current to stop it. Single photons thus produce a clear output signal that can be counted and timed. This output signal is typically independent of the number of photons producing electron-hole pairs in the depletion region, so two photons detected in quick succession will not increase the pulse height, and pile-up inspection based on pulse height [21,62] is not feasible.
SPADs and APDs have a much smaller active area than photomultipliers, typically between 10 and 100 µm in diameter. They can have larger quantum efficiency than photocathodebased photoelectronic vacuum detectors, up to about 80%. SPADs have a low dead-time and consequently high count rate, with transit time spread of 10s of ps comparable to MCPs [82,83]. However, they do have a charge carrier diffusion tail (a type of afterpulsing) in the IRF. Another characteristic feature of SPADs is their dark noise, typically tens of counts/s depending on the operating voltage and temperature [84] and is a key difference compared to photoelectronic vacuum devices for which 0.02 events cm −2 s −1 have been quoted [71]. For typical nanosecond fluorescence lifetime measurements in 50 ns time windows, this is not a key issue, but the dark noise consideration gains in importance as the detection window and fluorescence lifetime to be measured increase, and the count rate drops. It is also an important point to consider when enlarging the size of the light sensitive area in SPADs, as the dark noise increases with the area [84].

Single Photon Avalanche Diode (SPAD) Arrays
SPADs can be arranged into arrays for spectroscopy or imaging, and can be manufactured by complementary metal oxide semiconductor (CMOS) technologies [85]. One-dimensional arrays [86][87][88][89][90][91] can have a good fill-factor, since the light-sensitive regions can be adjacent, with the timing electronics placed away from the illuminated area. Two-dimensional SPAD arrays can be used as a camera for direct wide-field imaging, where each SPAD is a pixel that can independently perform photon arrival time detection via TCSPC [92][93][94]. One of the features of such an approach is that it yields enormous overall count rates of GHz [95]. In addition to TCSPC operation, they can be used in gated mode [96]. Two-dimensional arrays built for fast photon timing have a fill factor well below 100% (e.g., from ∼1% [97,98] to ∼50% [99]) because the majority of the area in each pixel is occupied by electronic circuits to perform the timing. Quantum efficiencies of around 60% have been reported [100], which compares favorably to photocathode-based devices. Large arrays of SPADs have been demonstrated to be suitable for TCSPC [95,101], and the development of these detectors is currently developing at a fast pace [102].

Silicon Photomultipliers (SiPM)
Silicon photomultipliers, also known as solid state photomultipliers, or multi-pixel photon counters (MPPCs) are arrays of hundreds to tens of thousands of integrated SPADs, each of which can detect photons individually and independently [103][104][105]. However, unlike in SPAD arrays, they are all connected to a common readout. Upon detection of a photon in a SPAD, an electrical signal is generated by the SPAD. This can be added to the signal from other SPADs that have detected a photon, to generate a pulse whose amplitude is proportional to the number of photons detected by the SiPM-this is the analog operation of the SiPM, which yields information about the photon flux. Alternatively, the number of SPADs that detected a photon can be counted separately, so that the number SPADs producing a signal is proportional to the number of incident photons, or the photon flux.
Interestingly, SiPMs have a photon number resolving capability: one, two, three or more detected photons can easily be resolved [103]. A pulse height distribution of the SiPM output shows characteristic peaks corresponding to the number of photons detected, in addition to picosecond arrival timing. The latter feature has been used in employing a SiPM with an active area of 1 mm 2 for TCSPC [106].
One feature of SiPMs and SPAD arrays is optical cross-talk. The detection of a photon by the semiconductor can create an infrared photon, which in turn can be detected by a neighboring SPAD [107][108][109]. However, while the effect exists in principle, it can be minimized with an appropriate SPAD design, and cross-talk as low as 1% has been reported [109].
SPAD arrays and SiPMs can be manufactured by CMOS technologies, but are usually produced using an ad hoc technology [85]. This is a photolithographic fabrication process using field effect transistors for logic functions in various integrated circuits, but also for analog circuits in image sensors; for example, the 160×128 SPAD array in Veerappan et al. [92] has been quoted to have 60 million transistors. While the latter SPAD array was manufactured using 130 nm CMOS technology, the latest SPAD arrays have been produced in 40 nm CMOS technology [110], allowing smaller features on the SPAD array. The pixel pitch, for example, 18.4 µm in x and 9.2 µm in y is the smallest one reported for SPAD arrays at the time of writing, much smaller than, for example, the 50 µm pitch used in Veerappan et al. [92]. Since the light-sensitive area in a pixel has to share the available space with the timing electronics, the fill-factor (i.e., the ratio of light-sensitive area to total pixel area) is less than 100%, if both are implemented in a two-dimensional layer. This is currently the case, but threedimensional integration, where the light-sensitive part and the electronics are implemented in separate layers which are then put together to form the SPAD array, or SiPM, is an area of active and rapid development, and 100% fill-factor devices will likely be produced in the near future. This would be a very welcome development for many application areas, including biophotonics [102,111]. Three-dimensional integration will also enable placement of FPGA-like (field programmable gate array) structures beneath the sensor, for detailed analysis of the photon flux, e.g., mean fluorescence lifetime determination without reading out the arrival time of every photon [112][113][114].

Hybrid Detectors
Hybrid photodetectors combine the PMT vacuum tube with photodiode semiconductor technology: a photoelectron liberated from the photocathode is accelerated through a high voltage (up to 10 kV) directly onto an avalanche diode, biased below the breakdown voltage (see Figure 6A) where it generates a large number of electron-hole pairs, one per each 3.7 eV [115]. Thus, well over 1,000 electron-hole pairs are created by a single photon, which are further amplified in the diode, see Figure 7. One advantage of this technology is that the single gain step produces a narrow pulse height distribution [115,116], which can be used to detect photon numbers. From the timing point of view, this single gain step results in low transit time spread and therefore low timing jitter [117].
Hybrid detectors feature lower dead-times (<1 ns) compared to either PMTs or SPADs (tens of ns). Also, the IRF of such detectors is, unlike that of a SPAD, PMT or MCP, free of afterpulsing or charge carrier diffusion tails, which reduces counting background and results in a considerably increased FIGURE 7 | In a hybrid detector, the photoelectron from the photocathode is accelerated through a high voltage into the p-n junction, where it creates a large number of electron-hole pairs. The gain is further increased by impact ionization, producing a timing pulse at the anode.
lifetime accuracy for a given number of detected photons [117,118].
Hybrid detectors are now the most commonly used detectors in point-scanning TCSPC FLIM applications. This is due to their high quantum efficiency for GaAsP photocathodes (∼50%), their large area (compared to SPADs) and fast time resolution (20 ps) [51]. Sixteen channel version of hybrid detectors are available. The principle behind hybrid detectors has also been used to create pixelated detectors for imaging (i.e., electron-bombarded CCDs [115,119,120] and CMOS cameras [121,122]) but these devices have not been modified for fast timing applications.

Superconducting Detectors
Superconducting single photon sensors are a relatively new technology. Superconducting nanowire single photon detectors, developed in 2001 [126], have potential for low jitter, short dead-time and high count rates [125]. In combination with TCSPC, an IRF of 2.7 ps has been reported, currently the shortest TCSPC IRF on record [127]. Transition edge sensors were first demonstrated in 1995 [128] and have 100 ns jitter and long deadtime making them unsuitable for fast timing applications [129]. The same is true for superconducting tunnel junction detectors, the single photon events have microsecond rise times, depending on the details of the superconducting material. They do have an intrinsic wavelength resolution, though, which is a very attractive feature. The biggest drawback of superconducting sensors is that they have to be operated at liquid helium temperature. Although they are an exciting development in the field of single photon detection, they currently find applications mostly in quantum optics, astronomy and long distance communications [129]. Consequently their development is aimed more toward the infrared wavelengths rather than visible light currently used in FLIM, to the best of our knowledge have not been used for infrared FLIM and are therefore out of scope for this review. For more information, see recent reviews [33,34,125,129].

Leading Edge and Constant Fraction Discrimination
After the electronic pulse created by a photon leaves the detector, it may pass through a pre-amplifier before it encounters some kind of discriminator. The discriminator sets the threshold for the pulse to be accepted, and shapes it for detection by the timing circuit. Leading edge discrimination sets a fixed threshold for the incoming pulse, and crossing this threshold provides the timing reference. If the pulses are of a constant amplitude, this is a feasible approach, for example for excitation laser trigger pulses. However, if the pulse height varies, then leading edge discrimination introduces an experimental uncertainty, time walk, which degrades the precision of the timing, as indicated in Figure 8A.
This time walk can be minimized by using a constant fraction discriminator (CFD). A CFD splits the input pulse in two, attenuates one of them, and delays and inverts the other, before they are added again, as shown in Figure 8B. The zero-crossing is independent of the pulse height. Therefore, for pulses for varying pulse height, as is typical for single photon detectors, a CFD minimizes the dependence of the photon arrival timing on the pulse height. The delay is chosen according to the rise time of the leading edge of the pulses to be timed.

Time-to-Amplitude Converter (TAC)/ Analog-to Digital Converter (ADC)
As the earliest photon timing technology, the TAC was derived from the delayed coincidence circuit [11]. It measures the time between a laser pulse and the arrival of a photon, and converts this time difference into a measurable voltage amplitude. A TAC typically starts charging a capacitor upon receipt of a "start" pulse, and stops charging it when a "stop" pulse arrives, such that the capacitor charge is proportional to the time elapsed between "start" and "stop." In practice, to avoid resetting the electronics after every laser pulse, TACs are usually operated in reversed start-stop mode where the arrival of the photon starts the TAC and the arrival of the next laser pulse stops it [130].
To record statistics of photon arrival times (e.g., fluorescence decays), the TAC output pulse, the height of which is proportional to the charge on the capacitor, is processed by an analog-to-digital converter (ADC). The ADC resolves the TAC voltage into a digital equivalent of the photon detection time, which is then used to address a memory location for the arrival time bin and increment it by one count. In early TCSPC systems the ADC was the bottleneck both in terms of speed and channel uniformity. This has been resolved by the development of electronics; the electronics required for TAC/ADC systems are relatively complicated, as the TAC requires a highly linear voltage ramp for precise operation, and the ADC has to be able to resolve the voltage into thousands of time channels with equal width [48,49]. The integral non-linearity of a TAC refers to the actual length of the time window compared to what it should be according to the settings. The differential non-linearity refers to the variation of the counts from time channel to time channel, and this is typically around 1%.
Nowadays TAC/ADC systems can achieve very fast timing in the order of a few ps. In this case the photon timing capabilities are usually limited by the timing jitter in the detector (i.e., transit time spread), and, with commonly used ps pulsed diode lasers, the optical excitation pulse width.

Time-to-Digital Converter (TDC)
TDCs are relatively new compared to TAC/ADC technology. As a fully digital system, a TDC counts clock cycles between a start and a stop pulse [131,132]. The time resolution of a TDC is determined by the clock speed, typically from a few tens to hundreds of ps at best (e.g., a 1 GHz clock has time resolution of 1 ns). In principle, TDC time resolution can be improved by using time interpolation circuits to measure clock cycle fractions, but in this case many advantages, such as simple circuit design and short dead-time are lost.
Although timing resolution of TDCs can be worse than those of TACs, they have some advantages over the TAC/ADC technology. TDCs can have low dead-times in the 1 ns regime, and are favorable in applications where low dead-times are a priority. TDCs can in principle measure any time span, unlike a TAC which has a limited voltage range, and are more robust against environmental changes (e.g., temperature) [53].
TDCs are much easier to upscale than TAC/ADC technology. For this reason TDCs have found widespread use in SPAD arrays, and the development of this technology is expected to continue. Recently a 32-channel TCSPC system has been developed employing the hybrid integration of a custom 32 SPAD array with 32-channel active quench and time to analog converter array [87, 133,134]. Although the fill-factor and SPAD performance are compromised by having such a large number of detectors and timing electronics on a single substrate, this detector has a TDC in each pixel with 55 ps resolution, allowing independent TCSPC in each pixel of a 32×32 pixel array simultaneously.
TAC/ADC systems, based on charging a capacitor with a constant current source [135], were and are mainly used in fluorescence spectroscopy systems, as they are a mature technology with the best timing precision and good linearity. However, they have a dead-time that does not make it feasible to time the arrival of two photons in one excitation cycle. TDC systems are based on clock cycles, and typically have a lower time resolution and higher differential and integral non-linearities than TACs, but they have a very short deadtime which allows timing of multiple photon arrivals after one excitation pulse [53]. They are also compact, and TDCs are typically the choice of timing circuit used on TCSPC-based SPAD arrays (although TACs have also been implemented [101]). For implementation in CMOS, on a SPAD array, a stable power supply, the power consumption and the heat generated by its operation, the temperature stability as well as non-uniformity across all pixels are all important considerations [135].

Streak Cameras
Streak cameras differ significantly from the other photon timing methods introduced in this review and are less common, but they are the fastest detectors available and have some uses in FLIM applications where extremely fast timing resolution is needed. Although they can photon count, they do not need TACs and ADCs or TDCs to perform the photon arrival timing, instead they convert photon arrival time into spatial position on a phosphor screen. They can be considered a more integrated or complete imaging system compared to PMTs, MCPs or SPADs which produce a pulse to be timed by appropriate separate electronics. Similar to image intensifiers, photon detection capability of a streak camera is based on electron amplification inside an MCP, but deflector plates and a voltage sweep circuit are placed between a photocathode and the MCP such that the photoelectrons liberated from the photocathode experience a deflecting voltage before hitting the MCP. The voltage experienced by the electron depends on the photon arrival time at the photocathode, therefore photons arriving at different times end up in different locations on the phosphor screen which is imaged by a camera, see Figure 9. This detection mode is often combined with spectral dispersion in orthogonal direction, such that the image has time distribution in one axis and spectral distribution in the other.
An obvious disadvantage is that one spatial dimension is taken up by the time axis, and the other possibly by spectral detection, so the camera is capable of measuring only one point at a time. It is possible to operate a streak camera in a line scanning mode, where the one spatial dimension in the camera images the time and the other one of the sample spatial directions, and a line is scanned across the sample for a twodimensional image [136,137]. Alternatively, a streak camera can be combined with a point-scanning confocal or two-photon microscope, where the time and spectral coordinates are obtained from the streak camera and the image is formed by usual pointscanning methods [138][139][140].
The operation of a streak camera is not necessarily based on single photon detection, although they can be operated in single photon mode [140,141]. The voltage sweep can be adjusted for the required time resolution and dynamic range-timing resolution in the sub-ps regime is possible with this method. Wider use of streak cameras is limited by their cost, also the time FIGURE 9 | Streak camera principle of operation. A photocathode converts photons into photoelectrons which pass through a pair of deflection plates, before being amplified and converted back into an optical image with a phosphor screen. Due to the deflection plates, timing information is converted into spatial information. Reproduced from Biskup et al. [139]. required for scanning an image and data processing to obtain a FLIM image can be lengthy.

FLUORESCENCE LIFETIME IMAGING
In FLIM microscopy, the timing of photons is used to build a histogram of photon arrival times-which is the fluorescence decay-in each pixel of an image, as schematically illustrated in Figure 2A. The decay is fitted with an exponential decay function (Equation 2) that yields the fluorescence lifetime, which is typically nanoseconds, and often depends on the microenvironment of the molecule, or the proximity of other fluorescent molecules. Therefore FLIM has found many applications, especially in the biological sciences, for monitoring the microenvironment and/or protein interaction [142][143][144][145].
When choosing a detector for FLIM, there is no simple answer for the "best" choice-this depends on the microscope and the sample. Fluorescence lifetimes are typically in the order of a few nanoseconds and in some applications IRFs of several hundred ps may be acceptable, but in order to measure complex multiexponential decays, or detect subtle changes in living specimens, fast timing of photon arrival is essential [51].
Apart from timing of individual photons, the time it takes to acquire a whole image is also very important in FLIM. For biological imaging and especially live cell applications, photon efficiency and light dose are critically important. Usually a compromise has to be found between timing accuracy, data acquisition speed and illumination light dose. FLIM methods can be generally divided into two categories: in scanning FLIM a laser beam is scanned over the image pixel by pixel and the image is recorded with a point detector (Figure 10A), whereas wide-field FLIM collects decays for the pixels in parallel fashion ( Figure 10B) [47,146].

Scanning FLIM
While the earliest fast fluorescence timing experiments were point measurements, usually measuring solutions in a cuvette, the development of laser scanning confocal microscopes [41] enabled TCSPC-based FLIM in the 1990s, and opened up new applications for FLIM in biological research. In scanning FLIM a laser beam is scanned over the image pixel by pixel, and the image is recorded with a point detector such as a PMT, SPAD or a hybrid detector. For proper analysis of the decay, a histogram is required for each pixel of the image, with good photon statistics in each pixel. The excitation repetition rate is fundamentally limited by the fluorescence lifetime to be measured so that the sample has time to decay to the ground state before a new excitation pulse, and to avoid photon pileup, the detection rate is usually limited to 1% of the excitation repetition rate at the brightest pixel. This can make image acquisition slow, typically a few minutes per image. Scanning FLIM is widely used, and it provides the benefits of confocal or multiphoton excitation microscopy, i.e., optical sectioning, reduced background blur and reduced photobleaching outside the focal point. Combination with spectral detection or polarization is also relatively straight-forward.
One limiting factor in acquisition speed is the dead-time of the detector and the timing electronics; this is the time required to reset the electronics, during which any arriving photon will be lost. PMTs can have dead-times of several tens of ns, while SPADs and hybrid detectors have improved this to <1 ns. On TAC/ADC based systems an upper limit to the detection efficiency is usually placed by the TCSPC electronics, which have typical dead-times in the order of 100 ns. TDCs have improved this to <1 ns, albeit with considerable loss of timing resolution [51].
Today, many commercial setups for scanning FLIM use hybrid detectors combined with TAC/ADC electronics. Hybrid detectors combine the advantages of both PMTs and SPADs: they have high gain, short dead-time, and low timing jitter. TAC/ADC electronics, on the other hand, have excellent timing resolution and can achieve IRFs down to 20 ps. While the electronics deadtime is usually in the order of 100 ns, the detection count rate is typically limited by the pile-up restrictions below 1 MHz.
It is possible to improve the overall image acquisition speed by using multiple detectors [147,148] to reduce the dead-time and distribute the photons over the detectors e.g., by spectral dispersion. Multi-point scanning with SPAD arrays has also been demonstrated [97,98] which enables real-time imaging of a FRET-based biosensor to study cell signaling, without trade-off between speed, noise and precision [149].

Wide-Field FLIM
In wide-field FLIM, the entire sample is illuminated to produce fluorescence, and the decay for each pixel is acquired in a parallel manner using some kind of camera, i.e., a position-sensitive detector. There are many different methods to perform wide-field FLIM, including time-gating and frequency-domain methods which are not based on timing single photons and are out of scope of this review-for more information see, for example, [20,150,151]. Image intensified fast cameras [152,153], as well as a timestamping camera, TimepixCam [154,155], have been used for timing single photons, but the photon timing resolution of these devices is limited from nanoseconds to microseconds, and thus they are more suitable for microsecond phosphorescence lifetime imaging rather than fast timing applications for nanosecond fluorescence decays. However, the latest TimepixCam version has 1.5 ns timing resolution [155] which would be suitable for measuring fluorescence decays of tens of nanoseconds.
For wide-field single photon timing on the picosecond time scale, MCPs and SPAD arrays are the most commonly used detectors. MCPs are a mature technology, and are used in many fields of science. Similar to PMTs, MCPs are capable of high timing resolution with IRFs down to 20 ps [79]. As they provide many channels, the position information is preserved during the signal amplification process. However, the MCPs themselves do not provide positional read-out, so they need to be combined with position read-out anodes [61,156,157]. Many different architectures have been designed [47], but usually the position read-out restricts the detection rate to a maximum of one photon per excitation pulse. Although advanced read-out architectures can improve this [158], overall this method does not offer significant improvement to the acquisition speed over scanning methods.
SPAD arrays are a more recent development in widefield single photon detection. Unlike MCPs where the position read-out often limits the detection rate to a maximum of 1 photon/excitation pulse, SPADs can count a photon/pixel/excitation pulse independently and therefore have potential for enormous count rates, and hence short acquisition times. The TDCs used in SPAD arrays can have an order of magnitude worse timing resolution than MCPs, but this is still acceptable for many FLIM applications where typical lifetimes are in the order of a few nanoseconds. The latest SPAD arrays can have fill factors of 50% [99], a great improvement over the megaframe SPAD arrays with 1% fill factor [97,98], but they generally do have some non-uniformity across the array. However, these specifications are continuously improving, as this technology is currently developing at a fast pace [102,110].
Wide-field photon counting methods are especially useful for combining FLIM with microscopy techniques that usually require a wide-field detector, such as TIRF [159] or lightsheet [160] illumination methods. They are also indispensable in live cell imaging, where the uniformly distributed, extremely low light dose enables long term observation of sensitive specimens [61,156,161,162].

APPLICATIONS
FLIM with fast timing has a vast range of applications in many different fields of science [143,163]. FLIM is often used for monitoring the microenvironment of living cells, such as pH [164], ion concentration (Ca 2+ , Cl − , K, Na,...) [165], viscosity [166,167], temperature [168,169], or oxygen levels [170,171]. FLIM is also the most precise methods for detecting FRET, which can be used for monitoring protein interaction, tension and folding [18]. A number of reviews and textbooks have delved into more details of the technique and its concepts, its implementations, applications and data analysis methods [20,48,49,51,151,[172][173][174][175][176][177][178][179][180][181].   representative fluorescence decays. The fluorescence lifetime difference between the images is due to selective excitation of the membrane-sensitive dye. Reproduced from Hirvonen et al. [159]. Typical fluorescence lifetimes are on the scale of a few nanoseconds which in principle can tolerate IRF widths of a few hundred ps. However, fast ps timing accuracy helps in detecting subtle changes in the microenvironment of living cells, or changes in FRET levels, especially in applications where the overall imaging speed is also a critical parameter [149].
Commercial scanning systems are now available that routinely achieve both fast photon timing accuracy, and fast overall image acquisition times. A confocal FLIM system has been reported with IRF FWHM <25 ps, which achieves count rates of up to 20 MHz [182]. The system uses a single detector but routes the photons into four separate timing channels, thereby shortening the TCSPC electronics dead-time. This system can obtain a 256 × 256 pixel FLIM image in about 0.5 s. This type of system has been used, for example, for monitoring the metabolic state in living cells [183], see Figure 11. This approach allows for an optical biopsy to be obtained, to assess cells for possible signs of disease, without employing mechanical methods for tissue removal or chemical methods for analysis. This so-called autofluorescence FLIM (because the sample does not need to be stained with fluorescence dyes, the intrinsic fluorescence of cells is used) is used for clinical diagnostics of skin [184]. Two lasers were multiplexed to alternatingly excite the endogenous fluorophores NAD(P)H and FAD which provide a read-out of the metabolic state, detected in two FLIM channels. Information on the metabolic state was derived from the fluorescence decays of NAD(P)H and FAD, which both have a fast few 100 ps and a slow ns scale component corresponding to bound and unbound fractions. A fast FLIM system is able to detect subtle changes in the component amplitude ratios and therefore changes in the metabolic state of the cells.
FLIM has also been used for monitoring viscosity in living cells using viscosity-sensitive fluorescent molecular rotor dyes [185], see Figure 12. Viscosity is the resistance of a fluid to flow, and is a key parameter determining chemical or biochemical reaction rates. Traditionally, bulk viscosity is measured by mechanical methods, but the use of fluorescent molecular rotors allows optical viscosity measurements. For this approach, only a very small sample is needed, and in combination with fluorescence microscopy and FLIM allows mapping viscosity on a micrometer scale, for example in living cells and their organelles [186].
The FLIM examples in Figures 11 and 12 are based on raster scanning a single point across the image, and performing TCSPC in each pixel separately, and, depending on the circumstances, it typically takes minutes to acquire a sufficient amount of photons for fluorescence decay analysis. The imaging speed can be improved by multi-point scanning. An 8 × 8 beamlet array was used to illuminate a sample, with a 32 × 32 pixel SPAD array detecting the emission from the spots excited by these beamlets with 55 ps timing resolution. Each beamlet scanned a 32 × 32 pixel area, producing a final image with 256 × 256 pixels. The original system had image acquisition time of 10 s / optical slice, the authors were able to detect FRET and calculate differences in cell biochemistry between sample [97,98]. An improved version was able to image four optical planes simultaneously with 4 × 4 beamlets in each optical plane, improving overall acquisition speed by a factor of 64 compared to conventional scanning FLIM [187]. With a more powerful laser, the generation of 32 × 32 beamlets could allow 8 × 8 beamlet acquisition of 16 individual planes simultaneously.
In wide-field TCSPC imaging, MCPs provide excellent timing resolution, but they need to be combined with a positionsensitive read-out [61,156,157]. Recently an MCP detector with picosecond timing resolution was combined with a delay line anode for position-sensitive read-out, and this detector combined with fluorescence microscopy techniques that are difficult to implement with scanning technologies [188].
TIRF microscopy is a method where an evanescent wave, created by reflection of an excitation beam via total internal reflection on a microscope cover slip, penetrates a few hundred nanometers into the sample. This technique is typically used to excite dyes or fluorescent proteins in or near membranes close to the coverslip, without creating background fluorescence from regions inside the cell. TIRF images are usually recorded with a camera, rather than raster scanning. Figure 13 shows a comparison of TIRF (left) and wide-field (right) fluorescence lifetime images of HeLa cells where the lifetime difference between the images is due to a membrane-sensitive dye [159]. The TCSPC-based wide-field FLIM approach for TIRF relies on the crossed delay line anode detector, which provides spatial and timing information for each detected photon.
Lightsheet fluorescence microscopy is another microscopy approach which is typically carried out using a camera. Figure 14 shows lightsheet FLIM images of cancer cell spheroids with two different fluorescent labels, one throughout the spheroid and one on the surface only [160]. Again, without a positionsensitive detector with picosecond time resolution, TCSPC-based lightsheet FLIM is hard to perform.

CONCLUSIONS
FLIM is a widely used imaging technique in the life sciences which allows not only the localization of fluorophores, but also the monitoring of their microenvironment and their interaction, typically via their nanosecond fluorescence decay. Photon counting methods allow the measurement of photon arrival time with picosecond time resolution. Both detectors and electronics play a critical role in fast timing of photons. From the detector side, transit time spread in the signal amplification process is a critical parameter for fast timing, whereas low dead-time and high detection efficiency are also important in fluorescence applications. For timing, the mature TAC/ADC technology can now achieve timing accuracy of a few ps, and is the most precise technology to use for fast timing applications. TDCs are a newer technology, whose timing capabilities are still limited to tens of ps by clock speeds, but they are easier to produce, more robust against environmental variations and easier to upscale, for example, to use in megapixel detector array technology for SPADs and SiPMs.
For fluorescence measurements, TCSPC has the best signal-to-noise ratio of the standard time-resolved imaging methods, and is accurate enough to allow multiexponential fluorescence decay fitting. The extremely low illumination intensity, distributed evenly over the field of view, is beneficial especially in life science applications where it allows long-term monitoring of living cells and organisms. The development of fast TCSPC methods will also benefit fields where single photon or particle time-of-flight measurements are required, for example LIDAR, neutron imaging ion velocity mapping or photon correlation techniques.

AUTHOR CONTRIBUTIONS
All authors listed have made a substantial, direct and intellectual contribution to the work, and approved it for publication.