Pursuit-evasion is an important problem in robotics with a wide range of applications including environmental monitoring and surveillance. Very often evaders are adversarial agents whose exact locations or actions are not known and can at best be modeled stochastically. Even when the pursuers are more capable and more numerous than the evaders, capture time may be highly unpredictable in such probabilistic settings. Optimization of time-to-capture in presence of uncertainties is a challenging task, and an understanding of how best to make use of the excess resources/capabilities is key to achieving that. This paper address the problem of assignment of pursuers to evaders and control of pursuers under such stochastic settings in order to minimize the expected time to capture.

We consider a multi-agent pursuit-evasion problem where, in a known environment, we have several surveillance robots (the pursuers) for monitoring a workspace for potential intruders (the evaders). Each evader emits a weak and noisy signal (for example, wifi signal used by the evaders for communication or infrared heat signature), that the pursuers can detect using noisy sensors to estimate their position and try to localize them. We assume that the signals emitted by each evader are distinct and is different from any type of signal that the pursuers might be emitting. Thus the pursuers can not only distinguish between the signals from the evaders and other pursuers, but also distinguish between the signals emitted by the different evaders. Likewise, each pursuer emits a distinct weak and noisy signal that the evaders can detect to localize the pursuers. Each agent is aware of its own location in the environment and the agents of the same type (pursuers or evaders) can communicate among themselves. The environment (obstacle map) is assumed to be known to either type of agents.

Each evader uses a control strategy that actively avoids the pursuers. The pursuers need to use an assignment strategy and a control strategy that allow them to follow the path with least expected capture time. The evaders and pursuers are aware of each others’ strategies (this, for example, represents real-world scenario where every agent uses an open-source control algorithm), however, the exact locations and actions taken by one type of agent (evader/purser) at an instant of time is not known to the other type (pursuer/evader). Using the noisy signals and probabilistic sensor models, each type of agent maintains and updates (based on sensor measurements as well as the known control/motion strategy) a probability distribution that random variable for evader position type (pursuer/evader) see Figure 1. In this paper we use a first-order dynamics (velocity control) model for point agents (pursuers or evaders) as is typically done in many multi-agent problems such as coverage control (Cortes et al., 2004; Bhattacharya et al., 2014) and artificial potential function based navigation Rimon and Koditschek (1992).

The main contributions of this paper are novel methods for pursuer-to-evader assignment in presence of uncertainties for total capture time minimization as well as for maximum capture time minimization. We also present a novel control algorithm for pursuers based on Theta* search (Nash et al., 2007) that takes the evaders’ probability distribution into account, and present a control strategy for evaders that try to actively avoid the pursuers trying to capture it. We assume that both groups of agents (pursuers and evaders) are aware of the control strategies employed by the other group, and can use that knowledge to predict and update the probability distributions that are used for internal representations of the competing group.

Section 3 provides the technical tools and background for formally describing the problem. In Section 4, we introduce the control strategies for the evaders and pursuers. In presence of uncertainties this control strategy becomes a stochastic one. We also describe how each type of agent predict and update the probability distributions representing the other type using this known control strategy. In Section 5, we present algorithms for assigning pursuers to the probabilistic evaders so as to minimize the expected time to capture. In Section 6 simulation and comparison results are presented.

Since the evaders are represented by probability distributions by the pursuers, the time-to-capture an evader by a particular pursuer is a stochastic variable. We thus consider the problems of pursuer-to-evader assignment and computation of control velocities for the pursuers with a view of minimizing the total expected capture time (the sum of the times taken to capture each of the evaders) or the maximum expected capture time (the maximum out of the times taken to capture each of the evaders). We assume that the number of pursuers is greater that the number of evaders and that the pursuers constitute a heterogeneous team, with each having different maximum speeds and different capture capabilities. The speed of the pursuers are assumed to be higher than the evaders to enable capture in any environment (even obstacle-free or unbounded environment). The objective of this paper is to design strategies for the pursuers to assign themselves to the evaders, and in particular, algorithms for assignment of the excess (redundant) pursuers, so as to minimize the total/maximum expected capture time.

While the evaders know the pursuers’ assignment strategy, they don’t know the pursuers’ positions, the probability distributions that the pursuers use to represent the evaders, or the exact assignment that the evaders determine. Instead, the evaders rely on the probability distributions that they use to represent the pursuers to figure out the assignments that the pursuers are likely using. We use a Markov localization (Thrun et al., 2005) technique to update the probability distribution of each agent.

Throughout this paper we use the following notations to represent the agents and the environment:

Configuration Space Representation: We consider a subset of the Euclidean plane, C ⊂ R 2 , as the configuration space for the pursuers as well as the evaders, which we discretize into a set of cells or vertices, V, where the agents can reside (Figure 1). A vertex in V will be represented with a lower-case letter v ∈ V, while its physical position (Euclidean coordinate vector) in C will be represented as X(v). For simplicity, we also use a discrete time representation.

Agents: The i ^th pursuer’s location is represented by r _i ∈ V, and the j ^th evader by y _j ∈ V (we will use the same notations to refer to the respective agents themselves). The set of the indices of all the pursuers is denoted by C r , and the set of the indices of all the evaders by C y .

Heterogeneity: Pursuer r _i is assumed to have a maximum speed of v _i , and the objective being time minimization, it always maintains that highest possible speed. It also has a capture radius (i.e., the radius of the disk within which it can capture an evader) of ρ _i .

The pursuers represent the j ^th evader by a probability distribution over V denoted by p j t : V → R + . Likewise the evaders represent the i ^th pursuer by a probability distribution over V denoted by q i t : V → R + . The pursuers maintain the evader distributions, { p j t } j ∈ C y , which are unknown to the evaders. While the evaders maintain the pursuer distributions, { q i t } i ∈ C r , which are unknown to the pursuers (see Figure 2). The superscript t emphasizes that the distributions are time-varying since they are updated by each type of agent (pursuer/evader) based on known control strategy of the other type of agent (evader/pursuer) and models for sensors on-board the agents.

The goal for our assignment strategy is to try to find the assignment that minimizes either the total expected capture time (the sum of the times taken to capture each of the evaders in C y ) or the maximum expected capture time (the maximum out of the times taken to capture each of the evaders in C y ). We assume that there are more pursuers in the environment than the number of evaders. The following subsection provides some fundamental definitions and tools that are used to describe and solve the optimal assignment problem in Section 5.

In the next sections we will describe the control strategy used by a pursuer that allows it to effectively capture the evader assigned to it, as well as the control strategy of an evader that allows it to move away from the pursuers assigned to it.

In Section 5, for designing the assignment strategy for the pursuers we will consider two metrics to minimize: 1) the total expected capture time, which is the sum of the times taken to capture each of the evaders, and, 2) the maximum expected capture time, which is the times taken to capture the last evader. While the actual assignment is computed by the pursuers and unavailable to the evaders, the evaders will estimate the likely assignment in order to determine their control strategy.

As mentioned earlier, we assume that both types of agents know all the strategies used by the other type of agents. That is, the pursuers know the evaders’ control strategy and the evaders know the pursuers’ control and assignment strategies. However the pursuers do not know the evaders’ exact position and vice versa. Instead they reason about that by maintaining probability distributions representing the positions of the other type of agents and update those distributions using the known control strategies of the other type of agents and weak signals measured by onboard sensors.

In presence of pursuers, an evader y _j actively tries to move away from the pursuers targeting it. With the evader at y ∈ V and deterministic pursuers, { r i } i ∈ I j , trying to capture it, we define a mean capture time as the harmonic mean of the capture time for each of the pursuers: τ ( y , { r i } i ∈ I j ) = 1 ∑ i ∈ I j 1 d ̃ g ( r i , y ) / v i (5) where d ̃ g ( r i , y ) = max 0 , d g ( r i , y ) − ρ i is the effective geodesic distance between r _i , y ∈ V (with d _g (r _i , y) being the geodesic distance or shortest path length between r _i and y), which accounts for the fact that pursuer r _i has a capture radius of ρ _i . For a given set of pursuer positions, τ thus a function that has higher value on the vertices in V that are farther away from the pursuers in I _j . The reason behind taking harmonic mean is that the harmonic mean gets lower contribution from distant pursuers and higher contribution from the nearby pursuers.

In order to determine the best action that the evader at y′ ∈ V can take, it computes the marginal increase in τ if it moves to y ∈ V (Figure 4): Δ τ ( y , y ′ , { r i } i ∈ I j ) = max 0 , τ ( y , { r i } i ∈ I j ) − τ ( y ′ , { r i } i ∈ I j ) + ϵ (6) where ϵ is a small number that gives a small positive marginal increase for some neighboring vertices in scenarios when the evader gets cornered against an obstacle.

A pursuer, r _i ∈ I _j , pursuing the evader at y _j needs to compute a velocity for doing so.

In a deterministic setup, if the evader is at y _j ∈ V, the pursuer’s control strategy is to follow the shortest (geodesic) path in the environment connecting r _i to y _j . This controller, in practice, can be implemented as a gradient-descent of the square of the path metric (geodesic distance) and is given by v i = k ∂ d g ( r i , y j ) 2 ∂ X ( r i ) = 2 k d g ( r i , y j ) z ̂ r i , y j , where k is a proportionality constant, d _g (r _i , y _j ) is the shortest path (geodesic) distance between r _i and y _j , and z ̂ r i , y j is the unit vector to the shortest path at r _i (see Figure 5). Such a controller does not suffer from local minimas due to presence of non-convex obstacles since the geodesic paths go around obstacles. A formal proof of that and the fact that ∂ d g ( r , y ) ∂ X ( r ) = z ̂ r , y , appeared in (Bhattacharya et al., 2014).). This gives a simple velocity controller for the pursuer.

In order to determine an initial assignment A 0 ⊆ F such that exactly one pursuer is assigned to an evader (thus potentially allowing unassigned pursuers).

Since for every ( i , j ) , ( i ′ , j ′ ) ∈ A 0 , T _ij and T _i′j′ are independent variables, the problem of finding the optimal initial assignment that minimizes the total expected capture time becomes ² A 0 = arg min A ′ ⊂ F s.t. ( i , j ) , ( i ′ , j ′ ) ∈ A ′ ⇒ i ≠ i ′ , j ≠ j ′ E ∑ ( i , j ) ∈ A ′ T i j = arg min A ′ ⊂ F s.t. ( i , j ) , ( i ′ , j ′ ) ∈ A ′ ⇒ i ≠ i ′ , j ≠ j ′ ∑ ( i , j ) ∈ A ′ E T i j (11) Thus, for computing the initial assignment, it is sufficient to use the numerical costs of C i j = E T i j in the assignment of pursuer i to evader j, and thus find an assignment that minimizes the net cost. In practice we use a Hungarian algorithm to compute the assignment. While a Hungarian algorithm is an efficient method for computing the assignment that minimizes the expected total time of capture, generalizing it to the problem of minimizing the expected maximum capture time is non-trivial, which we address next.

After computation of an initial assignment, A 0 , we determine the assignment of the remaining pursuers using the method proposed in (Prorok, 2020). Formally, we first consider the problem of selecting a set of redundant pursuer-evader matchings, A ̄ , that minimizes the total expected travel time to evaders, under the constraint that any pursuer is only assigned once: A ̄ = arg min A ′ ⊂ F s.t. ( i , j ) , ( i ′ , j ′ ) ∈ A ′ ∪ A 0 ⇒ i ≠ i ′ ∑ ( i , j ) ∈ A ′ E ( T i j ) . (12) Notably, the work in (Prorok, 2020) shows that a cost function such as (12), which considers redundant assignment under uncertain travel time, is supermodular. It follows that the assignment procedure can be implemented with a greedy algorithm that selects redundant pursuers near-optimally ³ .

Algorithm 2 summarizes our greedy redundant assignment algorithm. At the beginning of the algorithm, we sample h | C r | × | C y | -dimensional points from the joint probability distribution of { T i j } i ∈ C r , j ∈ C y and store them in the set T ̃ . In practice, the sampling is performed by sampling points, y _j ∈ V, from the evaders’ probability distributions, p _j , for all j ∈ C _y . The travel times, τ i j = 1 v i d g ( r i , y j ) , i ∈ C r , j ∈ C y then give the sample from the joint probability distributions of { T i j } i ∈ C r , j ∈ C y due to Eq. 4. The z ^th sample is thus a set of travel times between every pursuer-evader pair, and will be referred to as T ̃ z = { τ i j z } i ∈ C r , j ∈ C y ∈ T ̃ .

In this algorithm, we first consider the initial assignment, A 0 , and collect all the sampled costs of edges incident on to the j ^th evader into the variable S. Note that a given j ∈ C y appears in exactly one element of A 0 , thus the assignment in Line 4 assigns a value to a S j z exactly once. The set A ̄ contains the assignment of the remaining/redundant pursuers, that we initiate with the empty set.

In Line 10, we loop over all the possible pursuer-to-evader pairings, (i, j), that are not already present in A 0 or A ̄ , and which, along with A 0 or A ̄ , constitute a valid assignment. We go through all such potential pairings, (i, j), and pick the one with the highest marginal gain, T curr ⋆ − T new ⋆ . The pair with the highest marginal gain, is thus added to A ̄ . This process is carried out | C r | − | C y | times, thus ensuring that all pursuers get assigned.

One way that the inequality condition in Line 13 gets violated is when the marginal gains T _curr − T _new and T curr ⋆ − T new ⋆ are equal. This can in fact happen quite often when one or more redundant pursuers are left to be assigned and all of them are far from all the evaders, rendering marginal gains for any of the assignments close to zero. In that case a pursuer i gets randomly assigned to an evader j based on the order in which the pairs ( i , j ) ∈ F − A 0 − A ̄ are encountered in the for loop of Line 10.

In order to address this issue properly, we maintain a list of “potential assignments” that corresponds to (i, j) pairs (along with the corresponding T _new values maintained as an associative list, P\relax \special {t4ht=}A ⋆ ) that produce the same highest marginal gains (i.e., in line 13 equality holds), and choose the one with the median T _new value for inserting into the assignment set in Line 19.

Knowing the assignment strategy used by the pursuers, but the pursuers represented by the probability distributions { q i } i ∈ C r , the evaders use the exact same assignment algorithm to estimate which pursuer is being assigned to it. The only difference is that in Algorithm 2 the elements in the input, T ̃ , are sample travel times that are computed by sampling points, r _i , from the probability distribution, q _i , for all i ∈ C r , and then computing τ i j = 1 v i d g ( r i , y j ) as before. The assignment thus estimated is used by the evaders in computing their control as well as for updating the pursuers’ distributions, { q i } i ∈ C r , as described in Sections 4.1.1 and 4.2.2 respectively.