Optimizing supercontinuum spectro-temporal properties by leveraging machine learning towards multi-photon microscopy

1 Introduction

Multi-photon excitation (MPE) microscopy has become an essential tool for studying biological matter as it allows selective imaging, considering the 4D (x-y-z-t) examination of biological living tissues (Zipfel et al., 2003). Unlike one-photon excitation fluorescence microscopy techniques, where the excitation wavelengths typically span the ultraviolet and visible region (Cheng, et al., 2002; Campagnola and Loew, 2003; Rodriguez et al., 2006; Evans and Xie, 2008), MPE microscopy is implemented using a longer wavelength range (i.e., visible and near-infrared) resulting in a reduction of scattering loss and photodamage of the tissue sample, while providing excellent sectioning effect to ensure sufficient spatial resolution. MPE microscopy has therefore been used to image living tissues with further penetration depth (Helmchen and Denk, 2005; Larson, 2011) and to accurately probe the quantitative structure and function of cellular activities with sub-micro resolution (Zhao et al., 2014; Borhani et al., 2019). Recently, thanks to the advance in high-power femtosecond lasers, MPE microscopy has for instance been successfully reported for in-vivo deep brain imaging with a penetration depth above the millimeter range (Kobat et al., 2011; Horton et al., 2013; Weisenburger et al., 2019; Wang et al., 2020; Wu et al., 2021).

Beyond multi-photon absorption (MPA), MPE microscopy also relies on nonlinear parametric processes, which involve both multiphoton absorption and nonlinear wave mixing. These phenomena strongly depend on the sample structural constitution, the laser beam characteristics (e.g., pulse peak power, wavelength, spatio-temporal coherence), but also on the properties of the selected fluorophores (e.g., multi-photon excitation cross-section) when using labelled samples (Xu and Webb, 1996; Larson, 2011; Lefort, 2017). For instance, a laser source with high pulse energy provides a high photon flux, so that to mitigate scattering loss and thus increase the reachable imaging depth. However, such high energy pulses will inadvertently increase tissue heating, therefore resulting in damaging effects of the living tissues and their associated biological processes. As a consequence, adapting the laser illumination properties to match the selected fluorophore excitation requirements (or any other related multi-photon imaging modality such as second- or third-harmonic generation (Sheppard, 2020)) is a necessary step to improve the penetration depth and resolution of MPE microscopy. Nevertheless, in reality, having a versatile and tunable laser to meet these versatile specifications is a challenge, limited by e.g., the narrow bandwidth of the gain medium used in fiber or bulk laser systems.

As a consequence, only a few of the vastly available excitable fluorophores whose MPA peaks match fixed laser wavelength emissions can be readily used for MPE microscopy. To circumvent this issue, other widespread approaches consist in using processes of harmonic generation and optical parametric oscillators (OPO) or amplifiers (OPA) to adjust the excitation wavelengths required for multi-photon imaging techniques (Chu et al., 2003; Kobat et al., 2011). The drawbacks of these methods are the inherently complex and expensive setups used for practical experimental implementations, usually paired with several difficulties to operate and the requirement for a special maintenance.

As an alternative, supercontinuum (SC) generation has been implemented for MPE microscopy. Indeed, SC sources can provide a broadband wavelength spectrum (i.e., from ultraviolet to mid-infrared) covering numerous MPA peaks of the fluorophores (Poudel and Kaminski, 2019) or being directly relevant for multiplexed spectroscopy (Labruyère et al., 2012; Falconieri et al., 2019).

For instance, Coherent Anti-Stokes Raman Scattering (CARS), which was first studied in the sixties (Maker and Terhune, 1965; Begley et al., 1974), is a four-wave mixing phenomenon based on the identification of molecular vibrational modes that allows the structural cartography of labelled-free biological samples. Later on, that concept has been developed both in collinear and non-collinear geometry for vibrational signature identification in the CH region and for studying organic liquids, respectively (Duncan et al., 1982; Zumbusch et al., 1999).

Multiplex-CARS (M-CARS) experiments, based on simultaneous excitation of all frequencies ranging from 0 to 4,500 cm⁻¹ (using a monochromatic pump beam and a large band Stokes wave), have been demonstrated in 2002, by mixing polychromatic and monochromatic laser sources (Müller and Schins, 2002). In this framework, supercontinuum sources appeared as the best way to implement M-CARS systems (Okuno et al., 2007; Klarskov et al., 2011; Shen et al., 2018). However, because of the group velocity dispersion encountered in optical fibers, laser sources with long pulses have been privileged to guaranty temporal overlap between all the frequencies constituting the continuum and the quasi-monochromatic pump signals (Okuno et al., 2007). Yet, broadband SC has been scarcely implemented (directly) for multi-photon imaging because the residual part of the SC spectrum may give rise to additional noise within the MPA signals, on top of bringing undesired photodamage to the sample.

In fact, almost all MPA applications, including M-CARS techniques, rely on extracting a restricted range or only a few of the desired excitation wavelengths from the SC spectrum by using optical filtering techniques (Poudel and Kaminski, 2019). In such a case, however, only a small portion of the SC is extracted and used to generate the fluorescence signal, thus discarding most of the SC energy. More importantly, in numerous SC sources, the requirement of generating an extremely broadband signal with sufficient power spectral density over the whole SC spectral coverage is associated with dramatic drawbacks in terms of power efficiency. Indeed, in most commercial sources, these SC are generated from relatively long pulses (typically nanosecond duration) to bring about sufficient energy for extreme spectral broadening. Interestingly, however, MPE fluorescence signals are intrinsically related to the peak power of the excitation thus making the use of long pulses not always ideal for multiphoton microscopy. In fact, while the ultimate optical excitation source may be up for debate, it is clear that a versatile and reconfigurable system is highly sought-after. In this framework, customizing the properties of SC generation to maximize the power and the temporal waveform shape at suitably selected wavelengths thus constitutes an essential requirement.

As a succinct reminder, it is worth noting that SC generation can be seen as the result of dispersive and nonlinear effects acting together in a complex fashion during pulse propagation, typically in a guided structure such as a fiber or an integrated waveguide (Dudley and Taylor, 2010). As such, it is thus possible to tune SC spectro-temporal properties by changing e.g. the fiber structure, the fiber material, or the parameters of the input laser pulses. The first two approaches, however, do not exhibit proper methods for “on the fly” control properties of SC generation without significantly modifying the setup. In contrast, controlling nonlinear propagation by changing the input pulse laser parameters (i.e., pulse duration, peak power, chirp) has been used as a convenient way to optimize SC generation for practical applications, in particular, due to the strong dependence and sensitivity of spectral broadening dynamics on the input laser source parameters (Veljković et al., 2019; Sylvestre et al., 2021). Interestingly, such input laser parameters may be automatically modified via active systems (such as amplifiers, programmable filters, modulators, polarization controllers) enabling the implementation of optimization ions algorithms and thus allowing for live and autonomous adjustment of the desired SC properties. Recently, there has been a growing interest in applying various machine learning strategies to predict and control propagation dynamics in multidimensional nonlinear systems (Genty et al., 2021). For instance, deep neural networks have been deployed to predict light transmission properties of multimode fibers for imaging techniques (Kakkava et al., 2019), or to reconstruct a distorted image from the intensity of the output speckle patterns in such fibers (Borhani et al., 2018). For nonlinear propagation, a feed-forward neural network was also used to accurately predict the temporal profile of red-shifted solitons (Salmela et al., 2020) and the more global nonlinear propagation dynamics occurring during SC generation (Salmela et al., 2021).

For the adjustment of optical pulse properties, multiple strategies can be envisioned, including widespread pulse shaping techniques, chirp and dispersion engineering, etc. (Weiner 2011) Among those, an innovative approach based on integrated photonics has been demonstrated over the last years, by using cascaded on-chip Mach-Zehnder interferometers (MZIs) in combination with machine learning algorithms. This strategy was for instance instrumental for the demonstration of picosecond waveform laser pulses being autonomously reconfigured (Fischer et al., 2021). In a similar framework, a genetic algorithm (GA) optimization was recently used for the formation of reconfigurable pulse patterns so that to maximize the intensity of selected SC wavelengths after nonlinear fiber propagation (Wetzel et al., 2018). Such pulse patterns were created via a femtosecond pulse and an integrated photonic chip made of cascaded MZIs: the suitable modification of MZI splitting ratios allows for the generation of temporally interleaved pulse replica and, consequently, the control of SC spectral properties after nonlinear broadening. Unlike single-pulse seeds, the use of pulse patterns provides additional degrees of freedom (similar to a multidimensional system) to optimize the output properties of the SC generation via a variety of nonlinear phenomena including both intra-pulse effects and complex inter-pulse interactions (Andresen et al., 2011; Wetzel et al., 2018).

However, from an experimental perspective, several key properties of MPE microscopy (such as multi-photon excitation lifetime and cross-section, optical toxicity, etc.) not only rely on the peak power, but also on the temporal waveform of the excitation pulses. In addition, the requirement for advanced temporal waveform control is further highlighted by the rise/fall time of the detectors, which is typically in the order of hundreds of picoseconds. For an improved applicability in MPE microscopy, there is therefore an urgent need for novel yet easy and scalable approaches allowing for the fine tuning and optimization of SC spectro-temporal properties beyond simple filtering and/or power spectral density enhancement.

In this article, we conduct a numerical study to assess the viability of temporal pulse pattern shaping for MPE microscopy. Specifically, we implement various machine learning strategies to maximize MPA signals and waveform properties at specific wavelengths in the SC spectrum. In particular, we show how the use of ultrafast pulse patterns generated from a cascaded MZIs chip can be leveraged to adjust the output temporal waveform for a variety of MPA processes. The on-chip MZIs are used as an analogous to a multidimensional pulse pattern processor to shape the temporal waveforms (before nonlinear propagation). In turn, this allows for tuning the complex nonlinear dynamics of pulse broadening toward applications in MPE microscopy. We point out that, considering SC generation in a highly nonlinear fiber (HNLF), this strategy gives rise to improved flexibility compared to single pulse seeds with reconfigurable properties: we demonstrate the shaping of output pulses with a variety of temporal waveforms for two-photon absorption (2PA), three-photon absorption (3PA), and conjoint 2PA-3PA. Taking advantage of machine learning, the desired MPA signals can be maximized, with the temporal profiles and coherence degree of the waveform suitably adjusted.

2 Materials and methods

2.1 Numerical modeling of nonlinear pulse propagation

For our numerical study, we use a split-step Fourier method to solve the general nonlinear Schrödinger equation (GNLSE), as given in Eq. 1:

$\frac{\partial \mathit{A}}{\partial \mathit{z}}+\frac{\mathit{\alpha }}{2}\mathit{A}-{\displaystyle \sum _{\mathit{k}\ge 2}\frac{{\mathbf{i}}^{\mathbf{k}+1}}{\mathit{k}!}{\mathit{\beta }}_{\mathit{k}}\frac{{\partial }^{\mathit{k}}\mathit{A}}{\partial {\mathit{T}}^{\mathit{k}}}\mathbf{=i}\mathrm{\gamma }\left(1+{\mathit{\tau }}_{\mathit{shock}}\frac{\partial }{\partial \mathit{T}}\right)\left(\mathit{A}\left(\mathit{z},\mathit{T}\right){\displaystyle \underset{-\infty }{\overset{\infty }{\int }}\mathit{R}\left({\mathit{T}}^{\prime }\right){\left|\mathit{A}\left(\mathit{z},\mathit{T}-{\mathit{T}}^{\prime }\right)\right|}^2\mathit{d}{\mathit{T}}^{\prime }}\right)}(1)$

where A(z,T) is the pulse envelope and $T=t-{\beta }_{1}z$ is the time coordinate in the comoving frame at the group velocity ${\beta }_1^{-1}$. The right-hand side of Eq. 1 describes nonlinear effects including Kerr and Raman effects in which γ is the nonlinear coefficient and the shock timescale τ_shock = 1/ω₀ accounts for self-steepening. The nonlinear response function $R\left(T^{\prime }\right)=\left(1-f_R\right)\delta \left(T^{\prime }\right)+f_R{h}_R\left(T^{\prime }\right)$ includes the instantaneous Kerr effect and delayed Raman contribution in which f_R = 0.18 accounts for the Raman fraction. The left-hand side of Eq. 1 describes linear effects including loss (α) and dispersion (β_k).

Note that this GNLSE approach is considered for numerically simulating pulse evolution in both a highly nonlinear fiber and the waveguides constituting our on-chip programmable delay line (PDL - see details below).

The vacuum noise (shot noise) is taken into account in our model in Eq. 1 by adding one photon with a random phase on each spectral bin of the input waveform (Dudley et al., 2006). Herein, we do not consider additional noise effects, such as the amplitude fluctuation of each laser pulse (i.e. laser technical noise), polarization noise, etc., which typically degrade the coherence of the output SC spectrum. However, we assume that the impact of such noise contributions is limited when considering short laser pulse durations in the femtosecond regime (Zhu and Brown, 2004; Genier et al., 2019). The coherence degree $\left|g_{12}^1\left(\lambda ,0\right)\right|$ of the filtered spectrum at the wavelengths selected for MPA is calculated with a fixed value of splitting ratio (so that to generate the same pulse pattern) but with 20 simulations with a random noise (i.e. vacuum noise with one photon with random phase per spectral bin). The average coherence $\langle \left|g_{12}^1\left(\lambda ,0\right)\right|\rangle$ is considered as the mean value of coherence integrated over the bandwidth of the filtered spectrum.

For spectral broadening, we consider the nonlinear propagation of optical pulses into 10 m of HNLF. The parameters are those retrieved from a home-made (single-mode) Ge-doped fiber available in our laboratory, similar to typical commercial HNLF with a zero dispersion in the C-band around 1,550 nm. Considering a central wavelength at 1,560 nm, the fiber has a nonlinear coefficient of 3.5 W⁻¹km⁻¹ and dispersion is located in the anomalous regime with β₂ = -2.15 ps²/km and β₃ = 0.0693 ps³/km. The fiber losses at wavelengths of 1,200 nm, 1,560 nm and 2,000 nm are taken as 60 dB/km, 1 dB/km, and 100 dB/km, respectively.

For the initial processing of the input pulse prior to HNLF injection (so that to generate versatile temporal patterns of multiple femtosecond pulses), we also consider the propagation of the initial laser pulses into our on-chip programmable delay line: the considered PDL structure comprises 8 cascaded unbalanced interferometers (with increasing delays) to split an initial single pulse into up to 256 individual pulses with 1 ps separation between two adjacent ones, as reported in (Wetzel et al., 2018) and (Fischer et al., 2021). Each interferometer is an unbalanced interferometric structure (made of two integrated optical waveguides, two 50:50 optical couplers and a phase shifter) to constitute a balanced MZI followed by a pair of unbalanced waveguides. Each of these allows for splitting the incoming pulse(s) and delaying its (their) replicas by only changing the MZI splitting ratio (i.e., by adjusting the relative phase difference between the waveguide arms of the MZI).

A complete modelisation of the pulse propagation in the PDL is carried out by considering two GNLSE simulations (one for each waveguide) which are periodically coupled at the location of the MZI waveguide couplers while considering the phase shift implemented on the respective MZI arms. For this on-chip pulse splitting process on the PDL, we assume that the light propagating in each integrated waveguide is linearly polarized and follows a purely single-mode guiding condition (in a TM polarization mode). The PDL waveguides are therefore considered based on realistic experimental conditions so that at 1,560 nm, the dispersion parameters are β₂ = -2.87 ps²/km and β₃ = -0.0224 ps³/km, the nonlinear coefficient is 233 W⁻¹km⁻¹ and the losses of 0.06 dB/cm remain marginal for the propagation in the ∼5 cm-long waveguide structure (Ferrera et al., 2008; Wetzel et al., 2018).

In our model, we consider ideal MZI response (with perfect extinction ratio) so that the splitting ratio of each interferometer can vary over the range 0–1, in which a value of 0 indicates that the whole incoming light is routed into the following “short arm” section of the unbalanced interferometer, and a value of 1 conversely indicates a routing into the following “long arm” of the interferometer. Changing the splitting ratio of each interferometer thus leads in shaping the waveform of the pulse patterns, as illustrated in Figure 1.

FIGURE 1

FIGURE 1. Temporal (top) and spectral (bottom) profiles of the pulse patterns generated from the on-chip PDL for various splitting ratios, when initially injected with a 100 fs pulse at 1 kW peak power: (A) and (D) show the equipartition of the input pulse energy into 256 pulses when the splitting ratio is 0.5 for each MZI; (B) and (E) show the generation of an arbitrary pulse pattern when the MZIs’ splitting ratio are set to [0.2, 0.4, 0.6, 0.8, 0.6, 0.4, 0.2, 0]; (C) and (F) show the case where the whole input pulse energy remains in the first pulse ‘slot’ when the splitting ratio is 0 for each MZI (hence following the shortest path throughout propagation in the on-chip PDL).

In this study, we numerically consider two cases of nonlinear propagation into our HNLF, using either 1) single pulse seeds with adjustable parameters or 2) reconfigurable pulse patterns as generated from our on-chip PDL. The initial laser pulses possess a Gaussian shape as shown in Eq. 2:

$A\left(0,T\right)=\sqrt{P_0}exp\left(-\left(1+iC\right)\frac{2ln\left(2\right)T^2}{T_0^2}\right)(2)$

where P₀ is the peak power, T₀ is the pulse duration (full width at half maximum - FWHM) and C is the chirp factor.

For single pulse seeds, the spectro-temporal properties of SC generation are arbitrarily modified by adjusting the input pulse parameters: we consider a broad range of peak powers spanning from 5 to 20 kW, a pulse duration over the range 50–200 fs, and a chirp factor adjustable from -10 to +10. We note that the pulse duration is here constrained to this 50–200 fs range to ensure a high degree of coherence of the output SC spectrum. The central wavelength of the laser source is 1,560 nm.

For reconfigurable pulse patterns, we define a set of constant parameters for the initial pulse (i.e., 100 fs pulse duration, 1 kW peak power, 1,560 nm central wavelength) injected into the on-chip PDL. The splitting ratio of each interferometer is modified to shape the temporal waveform of the pulse patterns at the PDL output. Subsequently, the generated temporal pulse patterns are amplified by 20 dB as typically expected from an erbium-doped fiber amplifier (EDFA) before injection into the HNLF for nonlinear propagation, see Figure 2.

FIGURE 2

FIGURE 2. Schematic representation showing the use of reconfigurable temporal pulse patterns, as generated from an on-chip programmable delay line (PDL), to seed nonlinear propagation into a highly nonlinear fiber (HNLF) and spectral broadening. The interferometers settings on the PDL can be readily adjusted via a variety of machine learning approaches to optimize the output optical waveform and selectively enhance multi-photon absorption (MPA) signals at wavelengths suitable for e.g. bio-imaging applications (see the color code).

In our simulations, we considered a simple and ideal response of the EDFA (i.e. a pure 20 dB amplification) in order to focus on the specific tunability provided by the PDL chip processing (rather than the conjunction of multiple complex effects) while maintaining a good computing efficiency for quickly generating the desired optimization dataset. However, if required for more accurate numerical modeling (or direct experimental comparison), a more realistic EDFA response (including amplified spontaneous emission noise, saturation effects, nonlinear and dispersive signal distortions, etc.) may be readily implemented, and would likely further expand the parameter space and overall reconfigurability provided by the setup.

2.2 Optimization parameters and target functions for multiphoton excitation microscopy

As an illustrative example of our approach for the versatile optimization of multi-photon microscopy, we explore the potential application of reconfigurable SC for generating optical signals at wavelengths suitable for typical MPA excitations. For instance, we consider two sets of spectral regions respectively compatible with two- and three-photon absorption within the near-infrared region. In particular, we selectively filtered broadband signals with central wavelengths at 1,133 nm and 1,700 nm, towards the excitation of 2PA and 3PA for fluorophores and endogenous fluorescent proteins (FPs) with a central wavelength of the fluorescence at 566 nm (e.g., Texas Red, mKate and tdKatushka2). We also considered the wavelengths at 1,200 nm and 1,800 nm for 2PA and 3PA processes with central wavelengths of fluorescence at 600 nm (e.g., Alexa Fluor 680), see color-coded wavelengths in Figure 2.

The selected fluorophores (Texas Red, Alexa Fluor) are extremely popular for several reasons: their low-cost, minimal aggregation issues, and low cytotoxicity. The FPs have been used to increase the brightness in biological settings as well as far-red emitters with substantial potential for deep in-vivo imaging (Miller et al., 2017; Ricard et al., 2018; Wang et al., 2018). Moreover, excitation pulses at these selected wavelengths ensure minimizing the losses of the MPA process, including both scattering loss and photon absorption by tissues (Miller et al., 2017). Such a low photon absorption enables a better penetration depth of multi-photon imaging with low input laser power and also mitigates deleterious thermal effects (i.e., tissue heating and photodamage) which can be extremely harmful to biological samples.

For simplicity, we here assume that the spectral bandwidth (FWHM) of the fluorescent spectra from both 2PA and 3PA for all selected fluorophores is 20 nm, and accordingly, the FWHM filtering bandwidth of the excitation pulses at 1,133 and 1,200 nm (i.e. 2PA excitations) is 40 nm, while the filtering bandwidth at 1,700 and 1,800 nm (i.e. 3PA excitations) is 60 nm. The excitation pulses are extracted from the SC spectrum (in the frequency domain) using a Gaussian filter function, and the resulting filtered waveforms are retrieved in the time-domain by a Fourier transform.

The target functions shown in Eq. 3 are used for the optimization process. They correspond to the filtered intensity squared for 2PA (cubed for 3PA) and integrated with respect to time

$I_{2PA,{\lambda }_{\mathrm{1,2}}}={\displaystyle \int {\left(I_{{\lambda }_{\mathrm{1,2}}}\right)}^{2}dt}\text{and}{I}_{3PA,{\lambda }_{\mathrm{3,4}}}={\displaystyle \int {\left(I_{{\lambda }_{\mathrm{3,4}}}\right)}^{3}dt}(3)$

where λ_1,2,3,4 are 1,133 nm, 1,200 nm, 1,700 nm, and 1,800 nm, respectively. $I_{2PA,{\lambda }_{\mathrm{1,2}}}$ and $I_{3PA,{\lambda }_{\mathrm{3,4}}}$ correspond to the target functions related to the MPA signals for 2PA and 3PA, respectively.

The MPA signal according to the intensities in Eq. 3 is maximized by an optimization process based on evolutionary algorithms. In particular, we numerically implement two optimization techniques respectively based on a GA and particle swarm optimization (PSO), each possessing its own advantages (Jiang, et al., 2010; Fischer et al., 2021): GA optimization includes single-objective GA to maximize 2PA and 3PA signals, but also multi-objective GA functions for conjointly optimizing two individual MPA processes: 2PA-2PA or 3PA-3PA for two different fluorophores, as well as 2PA-3PA for a selected fluorophore. In the case of multi-objective GA, the optimization yields a Pareto front (also called Pareto frontier) that represents a trade-off between two MPA signal optimization.

In our GA model, the crossover fraction is set to 50%, and the population size to 1,000 individuals (with 3 genes for single pulse seeds and 9 genes for reconfigurable pulse patterns, respectively) (Katoch et al., 2021). The number of generation for multi-objective GA is set to 30. For single-objective GA, the minimum of the fitness function is typically found within 15 generations as shown in the following section. However, for several cases, the GA is operated with 20 or 30 generations to ensure that the GA algorithm would not lead to further improvement of the optimum fitness function.

For our PSO method (Bonyadi and Michalewicz, 2017), the optimization of 2PA and 3PA signals is implemented with a swarm size of 1,000 and a 9-dimension search space (the number of variables per particle to optimize–equivalent to the number of genes in a GA), the maximal number of generation is set to 40 or 60 (depending on the case) to ensure finding the optimal value of the fitness function, after which the algorithm is stopped (when it is not automatically interrupted beforehand when no further improvement of the optimum fitness function is observed).

We note that an experimental demonstration of SC spectro-temporal optimization falls out of the current study. However, such an experiment may be readily performed by measuring the output broadband waveform via spectro-temporally resolved characterization techniques (Weiner 2011). For instance, using (delayed) synchronized or asynchronous optical signals, cross-correlation based measurements such as frequency-resolved cross-correlation optical gating techniques (i.e. XFROG–see e.g. Wetzel et al., 2016) can be performed. This in turn allows to gather data compatible with fast iterative optimization of broadband SC signals in both the temporal and the spectral domain. While measurement noise and other experimental artefacts (e.g. detector sensitivity, acquisition time, laser drift, thermal/mechanical stability) may slightly hamper the repeatability, convergence speed or figure of merit of the optimization algorithm, the versatility of the proposed approach appears promising for experimental implementations relying on readily available characterization systems and optical measurement tools.

3 Results

3.1 Selective wavelength optimization with single pulse seed

The spectral broadening obtained via single femtosecond pulse propagation is induced by well-known soliton evolution dynamics in an anomalous dispersion regime. Such an evolution typically yields the following scenario: solitons are formed at the onset of fiber propagation via conjoint yet opposite dispersive and nonlinear effects. The perturbations from ideal (high-order) soliton dynamics - seeded by high-order dispersion, Raman scattering, attenuation, etc. - lead to soliton fission, so that the initial pulse breaks-up to form and eject multiple individual solitons. During further propagation, the ejected solitons are shifted toward longer wavelengths via soliton self-frequency shift (SSFS) - a result of Raman scattering. Correspondingly, the spectral components at shorter wavelengths, located in the normal dispersion regime, are generated by dispersive waves directly radiated at the trailing edge of the solitons. For example, Figures 3A,B presents the spectro-temporal evolution of a pulse with 12 kW peak power, 100 fs pulse duration, and chirp factor C = -5. The soliton fission occurs after around 30 cm of propagation, thus resulting in sudden spectral broadening. Soliton shifting contributes to further expanding the spectrum toward the longer wavelength until SSFS saturation is reached (Dudley et al., 2006). Dispersive waves are generated at wavelengths around 1,100 nm–1,200 nm. Any variation of the initial parameters (i.e., peak power, pulse duration, and chirp factor) brings about a noticeable alteration in the spectral broadening process, resulting in a change of MPA signals. However, SC generation induced by a single pulse seed provides only single-pulse-like MPA signals, as illustrated in Figure 3C.

FIGURE 3

FIGURE 3. Numerical simulation of SC generation in 10 m of HNLF using a single pulse (P₀ = 12 kW; T₀ = 100 fs ; C = -5). (A) Temporal evolution, (B) spectral evolution, (C) output spectrum and corresponding temporal waveforms filtered at selected wavelengths.

In order to assess the tunability of these MPA signals generated from SC generation, we have performed extensive simulations with adjustable input parameters. Figures 4B,C presents the evolution of maximal MPA signals (over successive sets of 1,000 simulations) obtained by single pulse seeds with varying initial parameters (i.e., peak power, pulse duration, chirp factor).

FIGURE 4

FIGURE 4. Evolution of maximal intensity for MPA signals with single pulse seeds: (A) 2PA excited at 1,133 nm, (B) 2PA excited at 1,200 nm, (C) 3PA excited at 1,700 nm, (D) 3PA excited at 1,800 nm.

We here compare numerical results obtained from GA optimization and Monte-Carlo method: Monte-Carlo simulations are implemented with 10⁵ random sets of initial parameters and we observe a progressive increase of MPA signals as we probe further the initial parameter space. GA optimization, on the other hand, brings on remarkable benefits: higher MPA signals can be readily obtained and, more importantly, for a given number of simulations, the MPA signals obtained from GA optimization are always better than random Monte-Carlo ones (making our GA approach computationally efficient, with a smaller number of simulations required to reach a desired MPA signal level). Of course, both the efficiency and convergence speed of GA optimization depend on the selected target wavelengths, and thus of the nonlinear dynamics involved. For instance, 3PA signals excited by 1,800 nm and enacted by soliton shifting are maximized with 4,000 simulations (see Figure 4D), while 2PA signals generated by 1,133 nm and induced by dispersive wave are rather maximized with 14,000 simulations (see Figure 4A).

3.2 Selective wavelength optimization with reconfigurable input pulse patterns

The spectral broadening obtained from multiple femtosecond pulses propagation is, comparatively, more complex than “standard” soliton evolution dynamics associated with single pulse evolution. Indeed, each individual pulse within a reconfigurable pulse pattern (as generated from the approach illustrated in Figure 5) possesses a different peak power and duration. In fact, depending on the splitting ratio of the on-chip PDL, respective pulses can exhibit different nonlinear dynamics for spectral broadening, yielding various interactions over propagation and, ultimately, leading to different contributions in MPA signal generation (in the pure sense of complexity).

FIGURE 5

FIGURE 5. SC generation in 10 m of HNLF with various reconfigurable input pulse patterns for the excitation of (A1–A3) MPA signals at all four selected wavelengths, (B1–B3) conjoint 2PA-3PA of a single fluorophore with excitation wavelengths at 1,200 nm and 1,800 nm, (C1–C3) conjoint 3PA-3PA of different fluorophores with excitation at 1,700 and 1,800 nm.

At the beginning of propagation, all individual pulses are spectrally broadened by self-phase modulation (SPM) and several ones, with sufficiently large peak powers (>62 W), will generate solitons due to the effects of dispersion and nonlinearity. Within them, pulses with higher peak powers (>245 W) can create a high-order soliton, which will subsequently break up into individual solitons via soliton fission. The ejected solitons have different durations, depending on the peak power of the individuals, and they respectively experience different self-frequency shift (i.e., solitons with higher powers would shift toward longer wavelengths), see Figure 5.

In such a scenario, complex inter-pulse interactions are expected to occur during further propagation: the temporal overlap between solitons ejected from different pulses, experiencing frequency shift at different rates, may occur at various fiber propagation distance. This can in turn lead to soliton collisions with energy exchange between the colliding solitons yielding the formation of new frequency components due to collision-induced dispersive waves (Luan et al., 2006; Erkintalo et al., 2010).

Noteworthy, even the contribution of low power input pulses within the initial temporal pattern (i.e. not yielding soliton formation) or the lower intensity components generated during propagation (i.e. radiated dispersive waves, etc.) may also play a key role in the nonlinear mixing dynamics, especially when there are overtaken by other faster/slower frequency components radiated from adjacent pulses.

In this framework, such collision dynamics can in principle be finely tuned by coherent control of the initial pulse patterns, so that to influence the power, duration, soliton frequency shift rate and relative phase between adjacent pulses, and thus adjusting the wavelength, propagation distance and timing of complex nonlinear conversion processes.

From a practical viewpoint, the adjustment of these multiple interactions may prove difficult. However, the suitable control of the initial parameters of the reconfigurable pulse patterns can provide a great variety and tunability to the excitation waveforms used for MPA.

For instance, a pulse pattern can provide MPA signals at all four selected wavelengths, yet in this case all excitation pulses filtered out from the SC essentially behave as single-like pulses with low peak powers, Figures 5A1–5A3. Other pulse patterns can be used for selective yet conjoint MPA processes: joint 2PA-3PA processes for a single fluorophore with an excitation wavelength at 1,200 nm and 1,800 nm can be achieved (Figures 5B1–5B3). Conversely, the excitation of dual 3PA processes (at 1,700 and 1,800 nm respectively) can be obtained so that to trigger a hybrid fluorescent medium including two distinct fluorophores (Figures 5C1–5C3).

The use of reconfigurable patterns brings on a flexible way to control the pulse shapes (i.e., temporal waveforms of the filtered pulses) toward a versatile reconfiguration and specific performance of the excitation pulse to be used for MPE microscopy.

As shown in Figure 6, while single input pulse seeds yield only single-like pulse waveforms for the desired wavelengths at the fiber output, the use of reconfigurable pulse patterns can offer both single-like pulse and more complex output waveforms with multiple pulse bunching operations. In fact, the delay and peak power of the individual pulses of the excitation waveforms can be effectively tailored by the modification of the input pulse patterns (i.e., only by changing the splitting ratio of the PDL). This brings about a potential to widen the modalities and selectivity for MPE and fluorescence lifetime microscopy.

FIGURE 6

FIGURE 6. Selected temporal waveforms of filtered pulses with single pulse seed and the reconfigurable pulse pattern at selected wavelengths: waveforms filtered at 1,133 nm (A1–A3), at 1,200 nm (B1–B3), at 1,700 nm (C1–C3), and at 1,800 nm (D1–D3). I_2PA and I_3PA signals are here given in µsW² and µsW³, respectively.

In Figure 6, we present examples of filtered output waveforms selected from Monte-Carlo simulations using random initial conditions (see Section 2.1). These post-selected results clearly illustrate the variety and reconfigurability brought about by using tailored input pulse patterns instead of single pulse with adjustable properties. However, we note that the intensities of the excitation pulses at the selected wavelengths can be readily maximized using different algorithmic approaches such as GA or PSO. These optimization algorithms enable obtaining high excitation intensities with a limited number of simulations, which is the approach we explore in the following sections. We also note that a comparison between these optimization techniques with advantages and limitations is further discussed in Section 3.4.1. Finally, it is worth mentioning that multiple pulse optimization is conducted via 256 pulses individually separated by 1 ps. This configuration is here well-suited for the propagation dynamics considered in the setup proposed in Figure 2. However, a larger number of pulses (increasing the dimensionality and potential variations in the parameter space) or a reduced number of pulses (yielding a faster convergence of the optimization algorithm) may also be considered depending on the constraints identified for experimental implementations. Similarly, depending on the dynamics involved during subsequent fiber propagation (e.g. initial pulse power, duration, repetition rate, fiber length), on-chip PDL with shorter or longer unit delay may be privileged to target particular optimization characteristics (e.g. temporal overlap of the initial pulses, emphasis on intra- or inter-pulse frequency conversion dynamics, coherence degree of the supercontinuum, etc.).

3.3 Selective multiphoton process and fluorophore optimization

The flexibility in terms of SC properties optimization, offered by reconfigurable input pulse patterns, allows in fact for customizing both the strength and trade-off between different MPA processes. For instance, a single broadband waveform can be used for the conjoint excitation of various MPA signals. This can be implemented for enhancing conjointly both 2PA and 3PA processes for a selected fluorophore, or conversely for the enhancement of dual 2PA-2PA or 3PA-3PA processes towards the combined excitation of two different fluorophores.

The results of such optimizations are presented in Figure 7, where we show the distribution of target MPA excitation intensities at selected wavelengths for various input pulse patterns. We compare the target function intensities (see Section 2.2) obtained by using either a multi-objective GA optimization (to find a corresponding Pareto front) or random simulations using a Monte-Carlo method.

FIGURE 7

FIGURE 7. Distributions and Pareto fronts of MPA excitation intensities obtained from multi-objective GA and random simulations at selected target wavelengths: (A) and (D) at 1,133 nm–1,200 nm, (B) and (E) at 1,700–1,800 nm, (C) and (F) at 1,200–1,800 nm. The blue shaded area (labeled “S”) illustrates a region of the Pareto front where the GA yields substantial improvements compared to Monte-Carlo simulations.

MPA excitation intensity distributions for different combinations of target wavelength are shown in Figures 7A–C. From the shape of these distributions (i.e. clustering at the bottom left corner), one can see that the optimization of dual wavelength objectives consists in a trade-off between two optimal points of single wavelength MPA signals. In other words, as the system energy is bounded from the initial conditions with constant power, it appears (unsurprisingly) difficult to obtain conjoint MPA process signals at the same levels as the ones obtained from their single MPA process optimization.

This general rule here applies regardless of the optimization technique, however, GA optimization is useful to reach similarly high target MPA excitation signals with a smaller number of simulations. More importantly, GA optimization enables to enlarge the Pareto fronts compared to the Monte-Carlo method, Figures 7D–F.

This aspect is important to tailor excitation waveforms and enhance the signals of conjoint MPA processes, which can be significant in several cases, as seen in the “S” shaded area in Figure 7A. Along with the potential to reconfigure individual excitation waveforms for distinct MPA processes, this approach is seen as an excellent way to achieve sophisticated characterization of various MPA processes and fluorophore combinations using the same experimental equipment and laser source.

To attest of this fine spectro-temporal waveform control, the filtered temporal profiles for conjoint MPA processes optimization are shown in Figure 8—here obtained for either single pulse or reconfigurable pulse patterns as input seeds.

FIGURE 8

FIGURE 8. Examples of temporal waveforms optimized by either single pulse seeds or reconfigurable pulse patterns for dual MPA processes enhancement at the selected wavelengths: (A1–A6) at 1,133 nm and 1,200 nm (B1–B6) at 1,700 and 1,800 nm (C1–C6) at 1,200 nm and 1,800 nm.

The use of single pulse seeds imposes a clear limitation for controlling the filtered MPA temporal profiles, typically associated with e.g. fixed relative delays between signals at selected wavelengths. In contrast, the MPA excitation waveforms are effortlessly tailored by using reconfigurable pulse patterns as input seeds. Using this approach, one can achieve the tunable adjustment of relative temporal delays between filtered signals, Figures 8A2–A3,B2–B3,C2–C3, but also enact more complex reconfiguration of the temporal waveforms to reach particular performances: selective pulse pattern adjustments can provide a variety of waveform excitations, for instance allowing for the formation of temporally-interleaved MPA signals, Figures 8A5–A6,B5–B6,C5–C6.

The ability of customizing the temporal waveforms for conjoint and/or selective MPA signals thus provides an innovative approach to develop and extend the potential for controlling complex multiphoton imaging techniques. Specifically, we expect that an improved tunability of the generated waveforms can prove useful to implement sequential spectral illuminations and time-encoded multiphoton processes using a versatile and scalable approach suitable for e.g. advanced STEAM or FLIM microscopy techniques. In a similar fashion, the use of reconfigurable output waveform bunches may mitigate several aspects associated with optical toxicity, by easily adjusting the equivalent peak power and repetition rate of the illumination (e.g. to alleviate either optical damage or thermal effects in the sample). While not explicitly mentioned, the use of selective filters along with suitable optimization techniques are expected to provide a way to generate excitation pulses with either uniform amplitude or adjustable relative power in various spectral regions of the output supercontinuum. Last but not least, tailoring the power spectral density and temporal waveform shape between different MPA processes may also be a way to indirectly adjust the penetration depth of a tissue sample. Indeed, it is known that 2PA processes for a selected fluorophore typically require less excitation power than ones of 3PA. However, with the use of a shorter excitation wavelength, 2PA yields higher scattering loss and thus lower penetration depth. Therefore, the idea of using reconfigurable pulse patterns for conjoint 2PA-3PA process optimization brings about a promising approach for “on-the-fly” imaging depth control (potentially without any moving parts).

3.4 Extension of optimization methods for multi-photon microscopy

3.4.1. Discussion on metaheuristic methods

We have seen that the use of reconfigurable pulse patterns gives rise to an extended parameter space so that to optimize a variety of MPA processes, similar to using an additional degree of freedom typical of multidimensional systems (here in the temporal domain, but analogous to e.g. the spatial dimension in multimode fibers). However, the efficiency of the targeted waveform intensity enhancement at selected MPA wavelengths depends strongly on both the algorithm used for optimization and the nonlinear dynamics involved during fiber propagation. Figures 9A,B presents the evolution of optimal excitation intensities for 2PA and 3PA, considering an optimization based on either PSO, GA, or Monte-Carlo methods (using 10⁵ random simulations). In the fiber propagation regime considered (see Section 2.1), the radiation of dispersive waves responsible for the generation of short wavelength components in a normal dispersion regime is less sensitive to the modification of the initial conditions than the generation of long wavelength components in the anomalous dispersion regime, here obtained via soliton shifting. In the former case (i.e. at 1,133 nm and 1,200 nm), metaheuristic optimization methods (GA or PSO) are not extremely significant compared to Monte-Carlo method to improve the target excitation intensity. In the latter case (i.e. at 1,700 and 1,800 nm), however, metaheuristic optimization techniques can prove very efficient to quickly reach optimal excitation intensities (at least twice higher than the ones from the Monte-Carlo method). For example, the maximal excitation intensity at 1,800 nm obtained by PSO optimization is 630 µsW³ after only 60,000 simulations (i.e. when the algorithm stops after convergence) while 10⁵ random simulations only provide a maximum intensity of 260 µsW³.

FIGURE 9

FIGURE 9. Evolution of maximal excitation intensity for MPA signals using reconfigurable pulse patterns, compared for different optimization approaches. (A) The intensity at 1,133 nm wavelength for 2PA, (B) at 1,200 nm for 2PA, (C) at 1,700 nm for 3PA, (D) at 1,800 nm for 3PA.

The results in Figure 9 also provide a comparison of optimization methods used to optimize MPA signals. In all the considered cases, GA reach an optimal value relatively quickly and within 10⁴ simulations (after which the GA optimization does not seem to provide further significant improvement). While these results may be finely tuned by the adjustment of the GA parameters (e.g. population size, crossover, etc.) to better explore the available parameter space, PSO can readily achieve better signal enhancement in all four tested cases, but at the expense of increased computing power and duration (i.e. larger set of simulations before reaching convergence).

The difference of efficiency between GA and PSO is caused by their principle (i.e., algorithm) to evaluate the best value for each iteration. GA relies on genetic evolution, and it starts with an initial parameter set of individuals (population). An iteration would be implemented if the termination criteria (e.g., number of generations) are not met so that the parameters of new individuals (i.e., children) are determined by interchanging gene groups of two selected current individuals (i.e., parents) through a crossover fraction (0.5 for our model). GA is usually a faster algorithm to find an optimum than other optimization algorithms, however, it does not guarantee to reach the global optimum of the system and may be stuck within a local minimum when exploring the parameter space.

PSO is, in contrast, modelled by the behavior of an animal swarm, starting with an initial swarm size of particles in a N-dimensional search space (here, N = 9). The position in the search space of the next particles (i.e., the particle’s trajectory) is updated by swarm intelligence that is determined by the current particle position, its individual-best position and the overall-best position of the particles in its neighborhood, following an algorithm regulated by self- and social-adjustment weights (Kennedy and Eberhart, 1995). As GA, PSO does not guarantee to converge towards the global optimum, an issue that can also be mitigated by the adjustment of the PSO parameters (e.g. larger swarm size, values of self- and social-adjustment weight, etc.).

However, while GA determines the next individuals only from the selected current individuals (with marginal mutation that can be further implemented), PSO creates the next particles position not only from current particles but also from individual-best and overall-best positions.

In other words, PSO essentially operates with higher degrees of freedom than GA, thus providing a greater diversity in the particle trajectories and exploration of the parameter space than GA. In particular, the momentum effects on the particle movements can provide a faster convergence when a particle is moving in a directional gradient. Such a behavior can be readily seen in our results, for instance through the exponential increase of the MPA intensities at 1,700 nm over the simulation iterations from 14,000 to 30,000, Figure 9C, and from 20,000 to 40,000 for MPA intensities at 1,800 nm, Figure 9D.

3.4.2. Coherence optimization for multi-photon microscopy

The coherence degree of the excitation pulses (i.e. the shot-to-shot stability in both phase and amplitude) is a significant factor when considering multiphoton imaging techniques, which potential impact strongly depends on the practical applications targeted. For example, stability on a short time scale is not necessary for general purpose fluorescence microscopy applications, since the acquisition time is much longer than the shot-to-shot fluctuation of the output spectra, and the results thus averaged over long timescales. In contrast, other applications, such as CARS, fast MPE microscopy (with high temporal resolution), as well as most pump-probe measurements involving different wavelengths or frequency conversion phenomena require excitation pulses with high stability, and may even need excellent coherence if an intrinsically phase-dependent process is involved.

It is well-known that the coherence of SCs generated in an anomalous dispersion regime are intrinsically linked to the peak power and duration of input pulses (Dudley et al., 2006). If a single input pulse is used, the degree of control on the system overall coherence is thus strongly limited. On the other hand, the use of a reconfigurable pulse pattern enables the adjustment of the coherence by modifying both the peak power and the potential interactions between each individual pulse during fiber propagation.

In such a case, we can therefore maximize the excitation intensities for 2PA and 3PA while optimizing the coherence of the spectral components filtered for MPA. These results are illustrated in Figures 10B,E, for which we run GA multi-objective optimizations trying to enhance MPA signals while either minimizing or maximizing the coherence degree of the corresponding filtered components.

FIGURE 10

FIGURE 10. (A) Pareto front of joint coherence and intensity optimization at 1,200 nm for 2PA, (B) Selected SC spectra with a high and low coherence for 2PA, (C) Corresponding spectra and filtered waveforms at 1,200 nm, (D) Pareto front of joint coherence and intensity optimization at 1,800 nm for 3PA, (E) Selected SC spectra with a high and low coherence for 3PA, (F) Corresponding spectra and filtered waveforms at 1,800 nm. The degree of coherence are shown in solid black line along the SC spectra of B, C, E and F (with scale on the right of the plots).

From the Pareto fronts of Figures 10A,D, respectively for 2PA and 3PA optimization, one can see that an excellent coherence degree close to unity (with $\langle \left|g_{12}^1\right|\rangle$ above 0.95) can be obtained in both optimizations with optimal MPA signals, while a lower value of coherence (with $\langle \left|g_{12}^1\right|\rangle$ ∼ 0.6) can also be achieved with less than a two-fold reduction of the corresponding MPA signal levels. These optimized waveforms, presented in Figures 10C,F, show that reconfigurable pulse patterns can indeed provide MPA excitation pulses with a similar intensity, but noticeably different coherence degrees and waveform shapes, opening further opportunity for accessing on-demand imaging modalities with versatile properties.

3.4.3. All-normal dispersion supercontinuum generation for multi-photon microscopy

For completeness, we also conducted similar optimization considering all-normal dispersion (ANDi) fiber propagation, a regime where spectral broadening is well-known for its inherent stability (Dudley et al., 2006; Sylvestre et al., 2021).

In our case, ANDi supercontinuum generation is simulated using Eq. 1 with the same input reconfigurable pulse patterns and parameters as those provided in Section 2.1 (i.e., pulse duration, peak power, amplification). For propagation, we consider a 2 m-long home-made fiber that has typical near-zero, flat, all-normal dispersion in both the visible and near-IR (Huang et al., 2018; Canh et al., 2020; Le et al., 2021). At 1,560 nm, our fiber has a nonlinear coefficient γ = 13 W⁻¹km⁻¹ and the dispersion coefficients are β₂ = 6.36 ps²/km, β₃ = 0.032 ps³/km, β₄ = 3 × 10^–4 ps⁴/km. In this regime, ANDi SC generation typically yields narrower spectral coverage than soliton-induced SC, requiring high peak powers for significant spectral broadening. As a consequence, we do not consider 2PA signals at 1,133 nm and 1,200 nm (that are too weak) for our optimization. However, for 3PA signals, multiple pulse optimization is also relevant for ANDi SC.

As shown in Figure 11A, the maximum values of excitation intensities at 1,700 and 1,800 nm for 3PA are enhanced with GA optimization to reach 81 (µW³) and 34 (µW³), respectively. While both signals can eventually be enhanced conjointly, as seen from the Pareto front in Figure 11B, we note that the MPA signals are much lower than the ones obtained through soliton-based dynamics presented in Section 3.4.1.

FIGURE 11

FIGURE 11. The excitation intensity filtered from ANDi SC for MPA. (A) Evolution of maximum excitation intensity at 1,700 and 1,800 nm for 3PA, (B) Pareto front of optimized intensities at 1,700 and 1,800 nm for conjoint 3PA processes.

Importantly, ANDi SC generation is mainly driven by SPM broadening at the onset of propagation, and followed by optical wave breaking (OWB). Consequently, ANDi SC typically consists of a flat-top output spectrum paired with a uniform pulse shape (in the time-domain) with a relatively simple temporal profile, instead of yielding the formation of highly-localized temporal structures (thus limiting the potential for the generation of complex and temporally interleaved output features). In the normal dispersion regime, modulation instability (the predominant phenomenon responsible for amplifying the vacuum noise during spectral broadening) is suppressed, resulting in output SC with significantly higher coherence (i.e., high shot-to-shot stability) than soliton-induced SC. However, ANDi regime here provides fewer degrees of freedom for efficiently controlling the spectro-temporal properties of SC generation, since OWB essentially occurs when there is a temporal overlap between SPM spectral components and the pulse tail (Heidt et al., 2011), so that during further propagation, no new spectral components are created. Therefore, the use of ANDi SC to maximize the excitation intensity for the targeted MPA processes is significantly limited.

4 Conclusion

Our results show a promising and feasible approach for controlling the spectro-temporal properties of excitation lasers for MPE microscopy: broadband SC generation covering numerous fluorophores absorption peaks (i.e. 2PA, 3PA) can provide a number of illumination wavelengths to match the proper excitation of selected fluorophores. By leveraging machine learning (GA, PSO), we can efficiently probe the input parameter space to selectively maximize particular output wavelengths, and thus rapidly enhance the desired MPA signals.

In particular, we show numerically that shaping an input wavepacket by means of an adjustable on-chip MZIs can be used to optimize a variety of temporal waveforms spectrally filtered from the output SC after propagation and spectral broadening. In fact, the on-chip MZIs can be used to create input pulse patterns with different properties (i.e., duration and peak power of the individual pulses) allowing for flexible tuning of the SC spectro-temporal properties for MPE microscopy. In contrast to single pulse seeds, reconfigurable pulse patterns provide a scalable and versatile way to adjust the output signals and temporal profiles at wavelengths directly useful for MPA processes (e.g. single-like pulses with high peak power, complex temporal profiles with multiple pulse bunching operations, etc.). From this approach, we demonstrate that the use of pulse patterns allows for controlling the delay and power between conjoint and interleaved MPA signals (e.g. conjoint 2PA-3PA processes for a single fluorophore, or conjoint 2PA or 3PA processes in a “mixture” of fluorophores).

Based on these promising numerical results, we expect that the practical implementation of these techniques will lead to further developments in MPE spectroscopy and multi-photon imaging techniques. With the flexibility to optimize excitation waveforms (in combination with machine learning), we anticipate that our approach will pave a way to enrich both the performances and modalities of MPE microscopy. While not exhaustive, such tailorable waveforms are seen as a possible way towards real-time control of MPE imaging depth, for conjoint multi-photon imaging techniques or tunable temporal interleaving of MPA signals (so that to observe living tissues and biological processes with minimal optical toxicity) or for direct image optimization and online analysis. It is worth noting that the dynamic illumination adjustment and combination of various degenerate and non-degenerate MPA processes holds a potential to probe spectral windows difficult to access from single wavelength excitation (e.g. low penetration depths from high scattering or water absorption regions) or to enhance the selectivity of particular modalities or fluorophore signatures towards adaptative image ‘segmentation’ of the sample content.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Author contributions

VH designed the study with contributions of BW. VH performed the core numerical simulations, the data analysis and wrote the manuscript. YB and LS contributed to numerical simulations and data analysis. SF and VC provided input and contributed to discussion. BW supervised and coordinated the project. All authors contributed to the writing and revision of the manuscript.

Funding

This work has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme under grant agreement No. 950618 (STREAMLINE project). BW further acknowledges the support of the French ANR through the OPTIMAL project (ANR-20-CE30-0004) and the Région Nouvelle-Aquitaine (SCIR & SPINAL projects).

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

Andresen, E., Dudley, J., Oron, D., Finot, C., and Rigneault, H. (2011). Nonlinear pulse shaping by coherent addition of multiple redshifted solitons. J. Opt. Soc. Am. B 28, 1716–1723. doi:10.1364/JOSAB.28.001716

CrossRef Full Text | Google Scholar

Begley, R. F., Harvey, A. B., and Byer, R. L. (1974). Coherent anti-Stokes Raman spectroscopy. Appl. Phys. Lett. 25, 387–390. doi:10.1063/1.1655519

CrossRef Full Text | Google Scholar

Bonyadi, M. R., and Michalewicz, Z. (2017). Particle swarm optimization for single objective continuous space problems: a review. Evol. Comput. 25, 1–54. doi:10.1162/EVCO_r_00180

PubMed Abstract | CrossRef Full Text | Google Scholar

Borhani, N., Kakkava, E., Moser, C., and Psaltis, D. (2018). Learning to see through multimode fibers. Optica 5, 960–966. doi:10.1364/OPTICA.5.000960

CrossRef Full Text | Google Scholar

Borhani, N., Bower, A., Boppart, S., and Psaltis, D. (2019). Digital staining through the application of deep neural networks to multi-modal multi-photon microscopy. Biomed. Opt. Express 10, 1339–1350. doi:10.1364/BOE.10.001339

PubMed Abstract | CrossRef Full Text | Google Scholar

Campagnola, P., and Loew, L. (2003). Second-harmonic imaging microscopy for visualizing biomolecular arrays in cells, tissues and organisms. Nat. Biotechnol. 21, 1356–1360. doi:10.1038/nbt894

PubMed Abstract | CrossRef Full Text | Google Scholar

Canh, T. L., Hoang, V. T., Van, H. L., Pysz, D., Long, V. C., Dinh, T. B., et al. (2020). Supercontinuum generation in all-normal dispersion suspended core fiber infiltrated with water. Opt. Mater. Express 10, 1733–1748. doi:10.1364/OME.395936

CrossRef Full Text | Google Scholar

Cheng, J., Volkmer, A., Book, L., and Xie, X. (2002). Multiplex coherent anti-Stokes Raman scattering microspectroscopy and study of lipid vesicles. J. Phys. Chem. B 106, 8493–8498. doi:10.1021/jp025771z

CrossRef Full Text | Google Scholar

Chu, S.-W., Liu, T., Sun, C.-K., Lin, C.-Y., and Tsai, H.-J. (2003). Real-time second-harmonic-generation microscopy based on a 2-GHz repetition rate Ti:sapphire laser. Opt. Express 11, 933–938. doi:10.1364/OE.11.000933

PubMed Abstract | CrossRef Full Text | Google Scholar

Dudley, J., and Taylor, J. (2010). Supercontinuum generation in optical fibers. New York: Cambridge University Press.

Google Scholar

Dudley, J., Genty, G., and Coen, S. (2006). Supercontinuum generation in photonic crystal fiber. Rev. Mod. Phys. 78, 1135–1184. doi:10.1103/RevModPhys.78.1135

CrossRef Full Text | Google Scholar

Duncan, M. D., Reintjes, J., and Manuccia, T. J. (1982). Scanning coherent anti-Stokes Raman microscope. Opt. Lett. 7, 350–352. doi:10.1364/OL.7.000350

PubMed Abstract | CrossRef Full Text | Google Scholar

Erkintalo, M., Genty, G., and Dudley, J. (2010). Experimental signatures of dispersive waves emitted during soliton collisions. Opt. Express 18, 13379–13384. doi:10.1364/OE.18.013379

PubMed Abstract | CrossRef Full Text | Google Scholar

Evans, C., and Xie, X. (2008). Coherent anti-stokes Raman scattering microscopy: chemical imaging for biology and medicine. Annu. Rev. Anal. Chem. Palo. Alto. Calif. 1, 883–909. doi:10.1146/annurev.anchem.1.031207.112754

PubMed Abstract | CrossRef Full Text | Google Scholar

Falconieri, M., Marrocco, M., Merla, C., Gagliardi, G., Rondino, F., and Ghezelbash, M. (2019). Characterization of supercontinuum generation in a photonic crystal fiber for uses in multiplex CARS microspectroscopy. J. Raman Spectrosc. 50, 1287–1295. doi:10.1002/jrs.5599

CrossRef Full Text | Google Scholar

Ferrera, M., Razzari, L., Duchesne, D., Morandotti, R., Yang, Z., Liscidini, M., et al. (2008). Low-power continuous-wave nonlinear optics in doped silica glass integrated waveguide structures. Nat. Photonics 2, 737–740. doi:10.1038/nphoton.2008.228

CrossRef Full Text | Google Scholar

Fischer, B., Chemnitz, M., MacLellan, B., Roztocki, P., Helsten, R., Wetzel, B., et al. (2021). Autonomous on-chip interferometry for reconfigurable optical waveform generation. Optica 8, 1268–1276. doi:10.1364/OPTICA.435435

CrossRef Full Text | Google Scholar

Genier, E., Bowen, P., Sylvestre, T., Dudley, J., Moselund, P., and Bang, O. (2019). Amplitude noise and coherence degradation of femtosecond supercontinuum generation in all-normal-dispersion fibers. J. Opt. Soc. Am. B 36, A161. doi:10.1364/JOSAB.36.00A161

CrossRef Full Text | Google Scholar

Genty, G., Salmela, L., Dudley, J., Brunner, D., Kokhanovskiy, A., Kobtsev, S., et al. (2021). Machine learning and applications in ultrafast photonics. Nat. Photonics 15, 91–101. doi:10.1038/s41566-020-00716-4

CrossRef Full Text | Google Scholar

Heidt, A. M., Hartung, A., Bosman, G. W., Krok, P., Rohwer, E. G., Schwoerer, H., et al. (2011). Coherent octave spanning near-infrared and visible supercontinuum generation in all-normal dispersion photonic crystal fibers. Opt. Express 19, 3775–3787. doi:10.1364/OE.19.003775

PubMed Abstract | CrossRef Full Text | Google Scholar

Helmchen, F., and Denk, W. (2005). Deep tissue two-photon microscopy. Nat. Methods 2, 932–940. doi:10.1038/nmeth818

PubMed Abstract | CrossRef Full Text | Google Scholar

Horton, N. G., Wang, K., Kobat, D., Clark, C. G., Wise, F. W., Schaffer, C. B., et al. (2013). In vivo three-photon microscopy of subcortical structures within an intact mouse brain. Nat. Photonics 7, 205–209. doi:10.1038/nphoton.2012.336

PubMed Abstract | CrossRef Full Text | Google Scholar

Huang, C., Liao, M., Bi, W., Li, X., Hu, L., Zhang, L., et al. (2018). Ultraflat, broadband, and highly coherent supercontinuum generation in all-solid microstructured optical fibers with all-normal dispersion. Photonics Res. 6, 601–608. doi:10.1364/PRJ.6.000601

CrossRef Full Text | Google Scholar

Jiang, H., Xie, K., and Wang, Y. (2010). Shooting algorithm and particle swarm optimization based Raman fiber amplifiers gain spectra design. Opt. Commun. 283, 3348–3352. doi:10.1016/j.optcom.2010.04.024

CrossRef Full Text | Google Scholar

Kakkava, E., Rahmani, B., Borhani, N., Teğin, U., Loterie, D., Konstantinou, G., et al. (2019). Imaging through multimode fibers using deep learning: The effects of intensity versus holographic recording of the speckle pattern. Opt. Fiber Technol. 52, 101985. doi:10.1016/j.yofte.2019.101985

CrossRef Full Text | Google Scholar

Katoch, S., Chauhan, S. S., and Kumar, V. (2021). A review on genetic algorithm: Past, present, and future. Multimed. Tools Appl. 80, 8091–8126. doi:10.1007/s11042-020-10139-6

PubMed Abstract | CrossRef Full Text | Google Scholar

Kennedy, J., and Eberhart, R. (1995). “Particle swarm optimization,” in Proceedings of IEEE International Conference on Neural Networks, 1942–1948. doi:10.1109/ICNN.1995.488968

CrossRef Full Text | Google Scholar

Klarskov, P., Isomäki, A., Hansen, K. P., and Andersen, P. E. (2011). Supercontinuum generation for coherent anti-Stokes Raman scattering microscopy with photonic crystal fibers. Opt. Express 19, 26672–26683. doi:10.1364/OE.19.026672

PubMed Abstract | CrossRef Full Text | Google Scholar

Kobat, D., Horton, N., and Xu, C. (2011). In vivo two-photon microscopy to 1.6-mm depth in mouse cortex. J. Biomed. Opt. 16, 106014. doi:10.1117/1.3646209

PubMed Abstract | CrossRef Full Text | Google Scholar

Labruyère, A., Tonello, A., Couderc, V., Huss, G., and Leproux, P. (2012). Compact supercontinuum sources and their biomedical applications. Opt. Fiber Technol. 18, 375–378. doi:10.1016/j.yofte.2012.08.003

CrossRef Full Text | Google Scholar

Larson, A. M. (2011). Multiphoton microscopy. Nat. Photonics 5, 1. doi:10.1038/nphoton.an.2010.2

CrossRef Full Text | Google Scholar

Le, H. V., Hoang, V. T., Canh, T. L., Minh, N. V., Kasztelanic, R., Klimczak, M., et al. (2021). Low pump power coherent supercontinuum generation in heavy metal oxide solid-core photonic crystal fibers infiltrated with carbon tetrachloride covering 930–2500 nm. Opt. Express 29, 39586–39600. doi:10.1364/OE.443666

PubMed Abstract | CrossRef Full Text | Google Scholar

Lefort, C. (2017). A review of biomedical multiphoton microscopy and its laser sources. J. Phys. D. Appl. Phys. 50, 423001. doi:10.1088/1361-6463/aa8050

CrossRef Full Text | Google Scholar

Luan, F., Skryabin, D., Yulin, A., and Knight, J. (2006). Energy exchange between colliding solitons in photonic crystal fibers. Opt. Express 14, 9844–9853. doi:10.1364/OE.14.009844

PubMed Abstract | CrossRef Full Text | Google Scholar

Maker, D. P., and Terhune, R. W. (1965). Study of optical effects due to an induced polarization third order in the electric field strength. Phys. Rev. 137 (3A), A801–A818. doi:10.1103/PhysRev.137.A801

CrossRef Full Text | Google Scholar

Miller, D., Jarrett, J., Hassan, A., and Dunn, A. (2017). Deep tissue imaging with multiphoton fluorescence microscopy. Curr. Opin. Biomed. Eng. 4, 32–39. doi:10.1016/j.cobme.2017.09.004

PubMed Abstract | CrossRef Full Text | Google Scholar

Müller, M., and Schins, J. M. (2002). Imaging the thermodynamic state of lipid membranes with multiplex CARS microscopy. J. Phys. Chem. B 106, 3715–3723. doi:10.1021/jp014012y

CrossRef Full Text | Google Scholar

Okuno, M., Kano, H., Leproux, P., Couderc, V., and Hamaguchi, H. (2007). Ultrabroadband (>2000 cm^−1) multiplex coherent anti-Stokes Raman scattering spectroscopy using a subnanosecond supercontinuum light source. Opt. Lett. 32, 3050–3052. doi:10.1364/OL.32.003050

PubMed Abstract | CrossRef Full Text | Google Scholar

Poudel, C., and Kaminski, C. (2019). Supercontinuum radiation in fluorescence microscopy and biomedical imaging applications. J. Opt. Soc. Am. B 36, A139–A153. doi:10.1364/JOSAB.36.00A139

CrossRef Full Text | Google Scholar

Ricard, C., Arroyo, E., He, C., Portera-Cailliau, C., Lepousez, G., Canepari, M., et al. (2018). Two-photon probes for in vivo multicolor microscopy of the structure and signals of brain cells. Brain Struct. Funct. 223, 3011–3043. doi:10.1007/s00429-018-1678-1

PubMed Abstract | CrossRef Full Text | Google Scholar

Rodriguez, L., Lockett, S., and Holtom, G. (2006). Coherent anti-stokes Raman scattering microscopy: A biological review. Cytom. A 69, 779–791. doi:10.1002/cyto.a.20299

PubMed Abstract | CrossRef Full Text | Google Scholar

Salmela, L., Lapre, C., Dudley, J., and Genty, G. (2020). Machine learning analysis of rogue solitons in supercontinuum generation. Sci. Rep. 10, 9596. doi:10.1038/s41598-020-66308-y

PubMed Abstract | CrossRef Full Text | Google Scholar

Salmela, L., Tsipinakis, N., Foi, A., Billet, C., Dudley, J., and Genty, G. (2021). Predicting ultrafast nonlinear dynamics in fibre optics with a recurrent neural network. Nat. Mach. Intell. 3, 344–354. doi:10.1038/s42256-021-00297-z

CrossRef Full Text | Google Scholar

Shen, Y., Voronin, A. A., Zheltikov, A. M., O’Connor, S. P., Yakovlev, V. V., and Sokolov, A. V. (2018). Picosecond supercontinuum generation in large mode area photonic crystal fibers for coherent anti-Stokes Raman scattering microspectroscopy. Sci. Rep. 8, 9526. doi:10.1038/s41598-018-27811-5

PubMed Abstract | CrossRef Full Text | Google Scholar

Sheppard, C. (2020). Multiphoton microscopy: a personal historical review, with some future predictions. J. Biomed. Opt. 25, 014511. doi:10.1117/1.JBO.25.1.014511

PubMed Abstract | CrossRef Full Text | Google Scholar

Sylvestre, T., Genier, E., Ghosh, A., Bowen, P., Genty, G., Troles, J., et al. (2021). Recent advances in supercontinuum generation in specialty optical fibers [Invited]. J. Opt. Soc. Am. B 38, F90–F103. doi:10.1364/JOSAB.439330

CrossRef Full Text | Google Scholar

Veljković, M., Mančić, A., Milović, D., and Maluckov, A. (2019). Numerical study of high intensity events in the supercontinuum generation in the presence of input chirp. Optik 196, 163180. doi:10.1016/j.ijleo.2019.163180

CrossRef Full Text | Google Scholar

Wang, T., Ouzounov, D., Wu, C., Horton, N., Zhang, B., Wu, C.-H., et al. (2018). Three-photon imaging of mouse brain structure and function through the intact skull. Nat. Methods 15, 789–792. doi:10.1038/s41592-018-0115-y

PubMed Abstract | CrossRef Full Text | Google Scholar

Wang, T., Wu, C., Ouzounov, D., Gu, W., Xia, F., Kim, M., et al. (2020). Quantitative analysis of 1300-nm three-photon calcium imaging in the mouse brain. eLife 9, e53205. doi:10.7554/eLife.53205

PubMed Abstract | CrossRef Full Text | Google Scholar

Weiner, A. M. (2011). Ultrafast optical pulse shaping: A tutorial review. Opt. Commun. 284, 3669–3692. doi:10.1016/j.optcom.2011.03.084

CrossRef Full Text | Google Scholar

Weisenburger, S., Tejera, F., Demas, J., Chen, B., Manley, J., Sparks, F., et al. (2019). Volumetric Ca2+ imaging in the mouse brain using hybrid multiplexed sculpted light microscopy. Cell 177, 1050–1066.e14. doi:10.1016/j.cell.2019.03.011

PubMed Abstract | CrossRef Full Text | Google Scholar

Wetzel, B., Bongiovanni, D., Kues, M., Hu, Y., Chen, Z., Trillo, S., et al. (2016). Experimental generation of riemann waves in optics: A route to shock wave control. Phys. Rev. Lett. 117, 073902. doi:10.1103/PhysRevLett.117.073902

PubMed Abstract | CrossRef Full Text | Google Scholar

Wetzel, B., Kues, M., Roztocki, P., Reimer, C., Godin, P.-L., Rowley, M., et al. (2018). Customizing supercontinuum generation via on-chip adaptive temporal pulse-splitting. Nat. Commun. 9, 4884. doi:10.1038/s41467-018-07141-w

PubMed Abstract | CrossRef Full Text | Google Scholar

Wu, J., Ji, N., and Tsia, K. (2021). Publisher Correction: Speed scaling in multiphoton fluorescence microscopy. Nat. Photonics 15, 87–812. doi:10.1038/s41566-021-00938-0

CrossRef Full Text | Google Scholar

Xu, C., and Webb, W. (1996). Measurement of two-photon excitation cross sections of molecular fluorophores with data from 690 to 1050 nm. J. Opt. Soc. Am. B 13, 481–491. doi:10.1364/JOSAB.13.000481

CrossRef Full Text | Google Scholar

Zhao, Y., Marjanovic, M., Chaney, E., Graf, B., Mahmassani, Z., Boppart, M., et al. (2014). Longitudinal label-free tracking of cell death dynamics in living engineered human skin tissue with a multimodal microscope. Biomed. Opt. Express 5, 3699–3716. doi:10.1364/BOE.5.003699

PubMed Abstract | CrossRef Full Text | Google Scholar

Zhu, Z., and Brown, T. (2004). Polarization properties of supercontinuum spectra generated in birefringent photonic crystal fibers. J. Opt. Soc. Am. B 21, 249–257. doi:10.1364/JOSAB.21.000249

CrossRef Full Text | Google Scholar

Zipfel, W., Williams, R., and Webb, W. (2003). Nonlinear magic: Multiphoton microscopy in the biosciences. Nat. Biotechnol. 21, 1369–1377. doi:10.1038/nbt899

PubMed Abstract | CrossRef Full Text | Google Scholar

Zumbusch, A., Holtom, G. R., and Sunney, X. X. (1999). Three-dimensional vibrational imaging by coherent anti-Stokes Raman scattering. Phys. Rev. Lett. 82, 4142–4145. doi:10.1103/PhysRevLett.82.4142

CrossRef Full Text | Google Scholar