Irreversibility in active matter: General framework for active Ornstein-Uhlenbeck particles

Active matter systems are driven out of equilibrium by conversion of energy into directed motion locally on the level of the individual constituents. In the spirit of a minimal description, active matter is often modeled by so-called active Ornstein-Uhlenbeck particles (AOUPs), an extension of passive Brownian motion where activity is represented by an additional fluctuating non-equilibrium"force"with simple statistical properties (Ornstein-Uhlenbeck process). While in passive Brownian motion, entropy production along trajectories is well-known to relate to irreversibility in terms of the log-ratio of probabilities to observe a certain particle trajectory forward in time in comparison to observing its time-reversed twin trajectory, the connection between these concepts for active matter is less clear. It is therefore of central importance to provide explicit expressions for the irreversibility of active particle trajectories based on measurable quantities alone, such as the particle positions. In this technical note, we derive a general expression for the irreversibility of AOUPs in terms of path probability ratios (forward versus backward path), extending recent results from [PRX 9, 021009 (2019)] by allowing for arbitrary initial particle distributions and states of the active driving.

Active matter systems are driven out of equilibrium by conversion of energy into directed motion locally on the level of the individual constituents. In the spirit of a minimal description, active matter is often modeled by so-called active Ornstein-Uhlenbeck particles (AOUPs), an extension of passive Brownian motion where activity is represented by an additional fluctuating non-equilibrium "force" with simple statistical properties (Ornstein-Uhlenbeck process). While in passive Brownian motion, entropy production along trajectories is well-known to relate to irreversibility in terms of the log-ratio of probabilities to observe a certain particle trajectory forward in time in comparison to observing its timereversed twin trajectory, the connection between these concepts for active matter is less clear. It is therefore of central importance to provide explicit expressions for the irreversibility of active particle trajectories based on measurable quantities alone, such as the particle positions. In this technical note, we derive a general expression for the irreversibility of AOUPs in terms of path probability ratios (forward versus backward path), extending recent results from [PRX 9, 021009 (2019)] by allowing for arbitrary initial particle distributions and states of the active driving.

I. INTRODUCTION
Irreversible thermodynamic processes are characterized by a positive entropy change in their "universe", i.e., in the combined system of interest and its environment [1]. In macroscopic (equilibrium) thermodynamics, where entropy is a state variable, this change usually refers to the difference between the entropy in the final state of the "universe" reached at the end of the process and in the initial state from where it started. In small mesoscopic systems on the micro-and nanometer scale, such as a colloidal Brownian particle diffusing in an aqueous solution, it has been established within the framework of stochastic thermodynamics [2][3][4][5][6] that the total entropy change should be evaluated from the entropy produced in the system and in its thermal environment along the specific trajectory the system follows during the process. This procedure remains valid even when the system is far from equilibrium, for example due to persistent currents or because it is driven by an external protocol realizing the thermodynamic process. The omnipresence of thermal fluctuations on the mesoscopic scale leads to a distribution of possible paths the system can take to go from the initial to the final state, and, accordingly to a distribution of entropy changes. A central result in stochastic thermodynamics is that the total entropy change ∆S along a specific realization of the system path (divided by Boltzmann's constant k B ) equals the log-ratio of probabilities for observing that specific path versus observing the same path in a time-reversed manner, i.e., traversing the same trajectory, but from the final state to the original initial state [2][3][4][5]. As a direct consequence, the total entropy change ∆S fulfills a so-called fluctuation theorem, exp(−∆S/k B ) = 1 (the angular brackets denote an average over all trajectories connecting the initial and final states), which can be viewed as a generalization of the second law of thermodynamics to the non-equilibrium realm when deviations from equilibrium are induced by externally applied forces or gradients.
A fundamentally different class of non-equilibrium systems are so-called "active particles", like Janus colloids with catalytic surfaces or bacteria [7][8][9][10][11], which have the ability to locally convert energy into self-propulsion, i.e., they move independently of external forces or thermal fluctuations. The source of non-equilibrium is the energy-to-motion conversion process on the level of the individual particle. This out-of-equilibrium process produces entropy, but the various degrees of freedom maintaining the self-propulsion are usually not observable in typical experiments with active particles, such that this entropy production can in general not be quantified. Moreover, for the (collective) behavior of active particles emerging from self-propulsion, as described, e.g., in [12][13][14][15], the details of the propulsion mechanism and the amount of dissipation connected with it are largely irrelevant. In analogy to the stochastic thermodynamics of passive Brownian particles, a central question in active matter is therefore how the path probabilities for translational degrees of freedom of the active particles and the associated log-ratio of forward versus backward path probabilities is connected to irreversibility and entropy production [16][17][18]. We remark that this is an ongoing debate [16,[19][20][21][22][23][24]] which we will not resolve here. Rather, we will provide a central step towards an understanding of the role of the path probability ratio in active matter by providing exact analytical expressions for a simple but highly successful and well-established [15,[25][26][27][28][29][30][31] model of active matter, namely the active Ornstein-Uhlenbeck particle (AOUP) [16, 18-21, 24, 32-44]. In this model, self-propulsion is realized via a fluctuating "driving force" in the equations of motion [7,10] with Gaussian distribution and exponential time-correlation (see Section II A). By integrating out these active fluctuations, we derive an explicit analytical expression for the path weight of an AOUP, valid for arbitrary values of the model parameters, arbitrary finite duration of the particle trajectory and arbitrary initial distributions of particle positions and active fluctuations (see Section IV). Using this path weight, we then derive the irreversibility measure in form of the log-ratio of forward versus backward path probabilities (Section V). Before establishing these general results, we briefly recall earlier findings from [18] for independent initial conditions of particle positions and active fluctuations, see Section III. We conclude with a short discussion in Section VI, including potential applications of our results.

A. Model
The model for an active Ornstein-Uhlenback particle (AOUP) consists in a standard overdamped Langevin equation for a passive Brownian particle at position x in d dimensions with an additional fluctuating force, which represents the active self-propulsion and which we denote by √ 2D a η(t), Here, the dot denotes the time-derivative, γ is the viscous friction coefficient, f (x, t) represents externally applied forces (conservative or non-conservative, and possibly time-dependent). Furthermore, ξ(t) are mutually independent Gaussian white noise sources modeling thermally fluctuating forces with δ-correlation in time, i.e., ξ i (t) = 0, ξ i (t)ξ j (t ′ ) = δ ij δ(t − t ′ ), and D is the particle diffusion coefficient, related to the temperature T of the thermal bath via Einstein's relation D = k B T /γ. All bold-face letters represent d-dimensional vectors with components usually labeled by subscripts i, j, etc. In analogy to the thermal fluctuations, we denote the strength of the active fluctuations η(t) by √ 2D a with an active "diffusion coefficient" D a . For an AOUP, the active fluctuations follow a Gaussian process with exponential time-correlations, which can be generated by a so-called Ornstein-Uhlenbeck process, where τ a is the correlation time of the active noise fluctuations, i.e.,

B. Central quantity of interest
Our central goal is to evaluate the path weight p[x | x i ] for particle positions alone, conditioned on the initial position x i for an arbitrary initial distribution p i (η i | x i ) of the active fluctuations given the specific value x i . By definition, we can write this path weight as where the path integral over η := {η(t)} τ f t=τ i includes the initial configuration η i , whereas the notation η := {η(t)} τ f t>τ i denotes the same history of active fluctuations without the initial configuration η i , and similarly for is the standard Onsager-Machlup path weight [45][46][47] for the joint process (x, η), where we use the shorthand notation v t = f t /γ = f (x(t), t)/γ and x t ≡ x(t), η t ≡ η(t), etc. The technical challenge consists in performing the integral over the active fluctuations η without explicitly specifying the initial distribution p i (η i | x i ).

III. PATH WEIGHT FOR INDEPENDENT INITIAL CONDITIONS
A. The results from [18] We start by summarizing the main results from [18]. In [18] we gave the path weight for trajectories x = {x(t)} τ t=0 , running from initial time τ i = 0 to final time τ f = τ , assuming that the active noise is initially independent of the particle positions and in its steady state, i.e., with the memory kernel where λ := 1 + D a /D/τ a and κ ± := 1 ± 1 + D a /D.

B. Stationary-state scenario
If we have a trajectory x = {x(t)} τ f t=τ i running from arbitrary times τ i to τ f instead, we can shift time as t −→ t − τ i and identify τ = τ f − τ i as the duration of the trajectory to convert (0, τ ] path weights to those running from τ i to τ f . Performing these replacements, the memory kernel (7) turns into Consequently, the corresponding path weight for a trajectory starting at x i at time τ i reads Letting τ i → −∞ (stationary-state scenario), the memory kernel becomes For "infinitely long" stationary-state trajectories, for which also τ f → ∞, this expression further reduces to The latter special case has been derived independently in [24] via Fourier transformation, see eq. (25) in [24], in order to analyze "entropy production" based on path-probability ratios. Similar Fourier-transform techniques for Langevin systems have been used in [48] for deriving a fluctuation relation at large times, with findings for the non-local "inverse temperature" as integration kernel in the "entropy production" corresponding to those in [24], and to our (11).

IV. PATH WEIGHT FOR ARBITRARY INITIAL CONDITIONS
In this section, we generalize the path weight (9) to allow for arbitrary joint initial distributions p i (x i , η i ) of particle positions and active fluctuations. Keeping in mind that we can time-shift final results between trajectories running during a time interval (0, τ ] and during arbitrary intervals (τ i , τ f ] as in Sec. III B, we here consider without loss of generality trajectories with τ i = 0 and τ f = τ . For notational simplicity we drop the subscripts (0, τ ] or [0, τ ] on p and Γ.
We start in Sec. IV A by first calculating Γ for a general Gaussian initial distribution of η 0 independent of x 0 , which has variance σ 2 and is centered atη 0 , Then, in Sec. IV B, we show how this result can be used to cover any arbitrary initial distribution p i (η 0 | x 0 ).

A. Gaussian initial distribution
With (4), (5), and the initial distribution (12), the path weight we want to evaluate reads The superscript ind emphasizes again that we use statistically independent initial conditions for x 0 and η 0 . After partial integration of theη t terms, similarly as in [18], we can express the path integral as with the differential operator Performing the Gaussian integral over η in (14), we obtain where Γ σ (t, t ′ ) denotes the operator inverse of V σ (t, t ′ ) in the sense that . It can be constructed similarly to the procedure in [18]. In particular, we can also , fixing the boundary terms as prescribed by (15). More details are given the Appendix. We find We note that (12) includes the steady-state distribution, p ss (η 0 ) = τ a /π e −τaη 2 0 , which arises for the active noise when evolving independently of the Brownian particle, as a special case for η 0 = 0 and σ 2 = 1/(2τ a ). Accordingly, we recover (6) and (7) when pluggingη 0 = 0 and σ 2 = 1/(2τ a ) into (16) and (17),

B. Arbitrary initial distribution
To cover arbitrary initial distributions In view of (5) we see that the term in brackets is exactly p ind η 0 ,σ [x | x 0 ] as defined in (13). Since we can also write p[ is the path weight conditioned on an initial position x 0 and initial state of the active noiseη 0 with arbitrary distributions.
With the explicit result (16) for p ind η 0 ,σ [x | x 0 ] we thus see that we have to calculate the σ → 0 limit of the expressions (σ 2 Det V σ ) 1/2 , [Γ σ (t, 0)/σ 2 − 1]/σ 2 , Γ σ (0, 0)/σ 2 and Γ σ (t, t ′ ). From (15) we observe that σ 2 V σ has a constant term (independent of σ) and contributions quadratic in σ such that (σ 2 Det V σ ) 1/2 reduces to an (irrelevant) constant as σ → 0. Next, setting t ′ = 0 in (17) and using τ a κ + − τ a κ − = 2τ 2 a λ we get If t = 0, too, we obtain such that Furthermore, we define as the memory kernel for the path weight conditioned on an arbitrary initial configuration (x 0 ,η 0 ) of particle positions and active fluctuations. Altogether, eq. (19) for this path weight then becomes where we have used κ + κ − = −D a /D in the third line. Finally, we can shift trajectories similarly as in Sec. III B to obtain the path weight for arbitrary trajectories x = {x(t)} τ f t=τ i conditioned on the joint initial state (x i , η i ) of position and active noise, with (26) Given an initial distribution p i (η i | x i ) of the active fluctuations conditioned on the initial particle position, we can then compute the position-only path weight of an arbitrary trajectory by averaging Equations (25), (26), (27) represent the first central result of the present contribution, a general expression for the path weight of active Ornstein-Uhlenbeck particles in position space only, for arbitrary trajectories with arbitrary initial and final times and arbitrary initial distributions. There is no approximation involved, so that our results are valid for any values of thermal and active noise parameters D and D a , τ a . We expect that the specific initial configuration becomes irrelevant for steady-state trajectories, i.e., in the limit τ i → −∞. As τ i → −∞, the second line vanishes in (25), because The third line enters into the integral over the initial configuration η i (see (27)) and thus decouples from the trajectory x resulting in an irrelevant prefactor. The only relevant contribution as τ i → −∞ is therefore the first line in (25) the same expression as Γ ind (−∞,τ f ] (t, t ′ ) from eq. (10). This illustrates that the system loses its memory about the initial state as τ i → −∞.
Another comparison to our previous results from Sec. III A [18] is obtained by plugging the stationary state distribution p i (η i | x i ) = p ss (η i ) = τ a /π e −τaη 2 i into (27) and performing the Gaussian integral over η i . In that case, we should get back the result (8), (9) for independent initial conditions. Indeed, including only the terms from (25) which involve η i , we evaluate the Gaussian integral over η i , yielding .

(30) A somewhat tedious but straightforward calculation then confirms Γ
, as expected.

V. IRREVERSIBILITY
In stochastic thermodynamics [2][3][4][5][6], irreversibility is quantified by comparing the probability t=τ i in a given experimental setup with the probabilityp[x] of observing the exact same trajectory traced out backwards when providing identical experimental conditions. In other words,p[x] is the probability of observing the "time-reversed" trajectoryx withx(τ i ) = x(τ f ) andx(τ f ) = x(τ i ), under the time-reversed experimental protocolf (x, t) := f (x, τ f + τ i − t) (note that we disregard for convenience the possibility that parts of the forces could be odd under time reversal; it is straightforward to adapt the expressions below accordingly if necessary). For passive Brownian motion, it has been shown that the log-ratio of these path probabilities is related to the dissipation occurring along the trajectory x, quantified as the total change of entropy in the thermal bath and the system. This fundamental connection makes the "irreversibility measure" a central quantity of interest also for active particles. Indeed, its connection with dissipation and entropy is under lively debate [16,[18][19][20][21][22][23][24]. We here provide a general expression for ∆Σ based on our result (25)- (27) for the path weight p[x]. Since the time-reversed trajectoryx is supposed to occur under identical conditions as the forward trajectory x, we can express its probability density via (25)- (27) as well, if we replace v(x, t) by the time-reversed protocolṽ(x, t) =f (x, t)/γ (see below eq. (31)). Using (31) we then rewrite the path weight for the reversed path in terms of the forward path (and the original protocol v t = v(x(t), t)). The resulting expression forp[x] is formally similar to (25), just with the sign inverted for allẋ(t) terms and all initial coordinates replaced by final ones. Plugging the path weights p[x] andp[x] into (32), and denoting the conditional average over the initial configuration η i of the active fluctuations in (27) by · η i | x i and the corresponding one over final configurations by · η f | x f , we find This expression constitutes the second central result of this work. Central properties of the active fluctuations which drive the particle motion are represented by the parameters D a (the strength of the active fluctuations) and τ a (their correlation time, hidden in λ = 1 + D a /D/τ a ). Moreover, our general result (33) contains averages over the distributions of the active fluctuations η i and η f at the beginning of the particle trajectory and at the beginning of the reversed trajectory (see also (27)). We therefore presuppose that we have some knowledge or control over these distributions when setting up the experiment, even though the (microscopic) degrees of freedom related to the active fluctuations typically are inaccessible, and so are specific realizations of η(t) or the specific values of η i and η f . For artificial active colloids [10], or in computer experiments, we may imagine, e.g., to let the particles orient randomly before "switching on" the activity, possibly with a specific strength (distribution).
In the spirit of quantifying irreversibility by asking how likely it is to observe a reversed trajectory compared to its forward twin when starting from identically prepared experimental setups (except for the initial particle position, which is x i for the forward path and x f for the backward path), we may take the distributions for η i and η f to be the same, or to be "mirror images" of each other under sign-inversion, depending on the physical situation modeled by the active fluctuations η(t) (see the discussion in [18]) [62]. Moreover, we may imagine the experiment to be prepared in a way that the initial distributions of the active fluctuations for forward and backward motion are independent of particle positions (a notable exception arising, if the experiment starts from a joint steady state). For such independent initial conditions with identical (or "mirrored") distributions, the third line in (33) vanishes. The second line, however, is still non-zero, and can be interpreted to quantify the contribution to irreversibility from the initial configuration of the active fluctuations.
The first line in (33) is independent of η i and η f , and thus measures the irreversibility associated with the time-evolution of the spatial particle position alone. It contains three terms (two in the double-integral and a boundary term), which all represent different contributions to irreversibility. The boundary term −k B ln[p(x f )/p(x i )] does not involve any parameters characterizing the thermal bath or the active fluctuations, and is usually interpreted as the change in system entropy of the AOUP between the beginning and end of the trajectory x [18]. The integral involving δ(t − t ′ ) is independent of the active parameters D a and τ a , and is formally identical to the entropy produced in the thermal bath along the trajectory x as known for passive Brownian motion [4]. However, in the present case of an AOUP it does not capture the full heat dissipation, because in addition to the force f (x(t), t) also active self-propulsion "forces" √ 2D a η(t) drive the particle and contribute to dissipation, i.e. even though the δ(t−t ′ )-integral can be interpreted as the "thermal contribution" to irreversibility due to the AOUP being in contact with a heat bath, it cannot be identified with the entropy produced in this thermal environment (see also the detailed discussion in [18]). The second, proper double-integral encodes the (statistical) characteristics of the active fluctuations via its kernel Γ [τ i ,τ f ] (t, t ′ ) and, furthermore, vanishes if active propulsion is "switched off", i.e. for D a = 0. Hence, it can be interpreted to measure the irreversibility "produced" by the active fluctuations along the trajectory x, and we will refer to it as the "active contribution" to irreversibility.
These two contributions to irreversibility from the particle trajectory x, the thermal one and the active one, are non-zero only if external forces f (x, t) = γv(x, t) are present in addition to the active self-propulsion (likewise for the integral in the second line, i.e. for the contribution associated with the initial preparation of the system), implying that the trajectories of "free active motion" appear reversible. For non-conservative forces, both contributions typically lead to a time-extensive increase of irreversibility with trajectory length τ . For conservative forces f (x) = −∇U (x) derived from a stationary confining potential U (x), the thermal contribution reduces to the boundary term [U (x i ) − U (x f )]/T and thus is non-extensive in τ . Due to the double-integral nature of the active part with the non-local kernel Γ [τ i ,τ f ] (t, t ′ ) a similarly obvious argument does not apply. Indeed, the question whether or not, or in how far, the trajectory of an AOUP in a confining potential appears (ir)reversible is still not fully answered [16,18,49]. We can, however, draw some conclusions from considering the limiting cases of small and large correlations times τ a → 0 and τ a → ∞. In the first case, τ a → 0, the active fluctuations become white and thus behave like a thermal bath, such that the AOUP can be imagined to be a passive Brownian particle in contact with a heat bath at effective temperature γ(D + D a )/k B , trapped in a confining potential. Accordingly, irreversibility production is non-extensive. In the second case, τ a → ∞, the active fluctuations become constant and thus behave like a bias force which slightly tilts the confining potential. Again, the situation is similar to a trapped passive Brownian particle with non-extensive irreversibility production. We can therefore expect that the active contribution to irreversibility in a confining potential may become maximal at some intermediate value of τ a .
Another important implication of the result (33) is that the rate at which irreversibility is produced in the stationary state (i.e., upon letting τ i → −∞, c.f. Section III B) is independent of the specific initial distribution p i (x i , η i ). Indeed, the terms in the second and third lines of (33) vanish as τ i → −∞, and the memory kernel Γ [τ i ,τ f ] (t, t ′ ) in the first line assumes the form (28), independent of the initial distribution (see the discussion in Section IV B). Hence, if we are only interested in long-time properties, the initial configuration in particular of the self-propulsion drive is irrelevant. While we might have intuitively expected that the long-time irreversibility production rate is independent of the details of the initial setup, it is not completely obvious in the presence of memory effects [50,51]. The fact that we can verify it for AOUPs is reassuring though, in so far as control over the initial state is limited in typical active particle systems (as already mentioned above). For infinitely long trajectories τ f → ∞ in the stationary state τ i → −∞, the expression for ∆Σ[x] reduces further to with Γ (−∞,∞) (t, t ′ ) from (11) (see the discussion around eq. (28)), in agreement with the findings in [24,48].
i.e., p i (η i | x i ) = p ss (η i ) = τ a /π e −τaη 2 i The resulting expression for (33) looks formally identical to the first line in (33) (8) As we can see from the calculation at the end of Section IV B above, the "amount of irreversibility" stemming from the particular stationary-state initial configuration of the active fluctuations has been absorbed into Γ ind [τ i ,τ f ] (t, t ′ ), and is therefore not explicitly visible in [18] as an additional term analogous to the second line in (33).

VI. CONCLUSIONS
What can we learn about the non-equilibrium nature of an active system by observing particle trajectories, i.e., the evolution of particle positions over time? Within the framework of a minimal model for particulate active matter on the micro-and nanoscale, the active Ornstein-Uhlenbeck particle [15,[25][26][27][28][29][30][31] (see eqs. (1), (2)), we here contribute an essential step towards exploring this question by deriving an exact analytical expression for the path weight (eqs. (25), (26), (27)), which is valid for any values of the model parameters, any external driving forces, arbitrary initial particle positions and configurations of the active fluctuations, and arbitrary trajectory durations. We use this general expression to calculate the log-ratio of path weights for forward versus backward trajectories (see eq. (33)). In analogy to the stochastic thermodynamics of passive Brownian particles [2][3][4][5][6], such an irreversibility measure may provide an approach towards a thermodynamic description of active matter [16,[18][19][20][21][22][23][24].
In future works we may build on these results to further explore the non-equilibrium properties of AOUPs. A highly interesting problem is a possible thermodynamic interpretation of the path probability ratio ∆Σ[x] [18], e.g., via exploring its connection to active pressure [14,52], to the different phases observed in active matter [53], or to the arrow of time [54,55] in these systems. Such a thermodynamic interpretation, in particular concerning the role of dissipation, may finally allow to quantify efficiency fluctuations in stochastic heat engines operating between active baths [56], in analogy to passive stochastic heat engines [57][58][59]. Other important questions which can be approached directly by using our general result for the path weight p[x] include the analysis of the response behavior under external perturbations [43] or of violations of the fluctuation-response relation [60,61] due to the inherent non-equilibrium character of active matter, and their potential for probing properties of the active fluctuations [61].