Consider two real-valued datasets { X t ̲ , Y t ̲ : t ̲ ∈ Z k } defined over a k-dimensional index set. These may in general be vector-valued random variables, and therefore X t ̲ and Y t ̲ can be considered as either univariate or multivariate time series, random fields or other types of structured data. Our problem of interest is where instances of X t ̲ and Y t ̲ match with certain localized motifs { X t ̲ : t ∈ S } ≈ { Y t ̲ : t ∈ S ′ } for small localized index sets S , S ′ ⊂ Z k . For both the univariate and multivariate cases, we can embed these data sets into some corresponding point clouds X , Y ⊂ R L via windowing with a size L window, where L can be determined from training or some other appropriate technique [8, 13]. Once we have a window embedding of these data sets, we can define various distance measures on the resulting point clouds to define similarity between X t ̲ and Y t ̲ .

A distance measure that has been proposed previously to determine similarity between two such time series embedded point clouds constructed over R L is MPdist [7]. In this case, a cross-data distance measure, denoted D ², can be constructed by using 1-nearest neighbor Euclidean distances between point clouds X and Y as below: d x = inf y ∈ Y ‖ x − y ‖ , ∀ x ∈ X d y = inf x ∈ X ‖ x − y ‖ , ∀ y ∈ Y D 2 = d 2 z : z ∈ X ∪ Y . (1)

In [7], the distance measure MPdist was estimated for univariate time series by choosing the kth smallest element in the set D ². In general, MPdist can be constructed using a lower quantile of the distance distribution D ².

Our proposed distance measure kdiff generalizes MPdist using a kernel-based construction, and by considering both cross-similarity and self-similarity. Similar to MPdist, we first construct sliding window based embeddings over the original data instances X t ̲ , Y t ̲ and obtain corresponding point clouds X and Y. For MPdist the final distance is estimated based on cross-similarity between the embeddings X, Y as shown in Eq. 1. Our distance measure kdiff differs from this in two ways:

• We use a kernel based similarity measure over the obtained sliding window based embeddings X and Y for kdiff instead of the Euclidean metric used in MPdist.

To define our kdff statistic, we will begin with a discussion of general distributions defined on R L . For the purposes of this paper, these can be assumed to be the distributions that the finite samples X are drawn from (in a non-iid fashion) and stitched together to form the time series X t ̲ (respectively for Y and Y t ̲ ).

In general, we can define the distributions on X , which is a locally compact metric measure space with the metric ρ and a distinguished probability measure ν*. The term measure will denote a signed Borel measure. We introduce a fixed, positive definite kernel K : X × X → ( 0 , ∞ ) , K ∈ C ( X × X ) . Since the kernel is fixed, the mention of this kernel will be omitted from notations, although the kernel plays a crucial role in our theory. Given any signed measure (or positive measure having bounded variation) τ on X , we define the witness function of τ by U τ z = ∫ X K z , x d τ x , z ∈ X , (2) and similarly the magnitude of the witness function T τ z = | U τ z | , z ∈ X . (3)

In the context of defining a distance between μ ₁, μ ₂, we take τ = μ ₁ − μ ₂, which results in a witness function U μ 1 − μ 2 z = E x ∼ μ 1 K z , x − E y ∼ μ 2 K z , y .

To quantify where T (μ ₁ − μ ₂) is small, we define the cumulative distribution function (CDF) of a Borel measurable function f : X → R by Λ f t = Λ ν * ; f t = ν * z : | f z | < t , t ∈ 0 , ∞ , (4) and its “inverse” CDF by f # u = sup t ∈ 0 , ∞ : Λ f t ≤ u , u ∈ 0 , ∞ . (5)

Both Λ(f) and f ^# are non-decreasing functions, and f ^# (u) defines the u -th quartile of f.

Finally, we are prepared to define our kdiff distance between probability measures μ ₁, μ ₂. Having defined T (μ ₁ − μ ₂) (z), we now define kdiff to be the α quantile of T (μ ₁ − μ ₂), k d i ff μ 1 , μ 2 ; α = T μ 1 − μ 2 # α , α ∈ 0,1 . (6)

The intuition of (6) is that, if μ ₁ = μ ₂, the resulting kdiff statistic will be zero. But beyond this, if T (μ ₁ − μ ₂) (z) = 0 for a set z ∈ A ⊂ X such that ν*(A) > 0, then for a localized enough kernel, there exists a quantile α for which we can still have the resulting kdiff statistic be close to zero. This allows us to match distributions that agree over partial support. This will be discussed more precisely in Section 2.3.

For the purposes of analyzing the kdiff statistic, we will focus on the setting of resolving mixture models of probabilities on X when only one of the components agree. Accordingly, for any δ ∈ (0, 1), we define P δ to be the class of all probability measures μ on X which can be expressed as μ = δμ _F + (1 − δ)μ _B , where μ _F and μ _B are probability measures on X . With the applications in the paper in mind, we will refer to μ _F as the foreground and μ _B as the background probabilities. Our interest is in developing a test to see whether given two measures μ ₁ and μ ₂ in P δ , the corresponding foreground components agree. Clearly, the same discussion could also apply to the case when we wish to focus on the background components with obvious changes. We also note that in general, P δ is simply the space of probability measures, but it should be thought of as the space of mixtures of almost-disjoint measures. We will add the necessary assumptions on δ, μ _F , μ _B in the main results below.

We first present some preparatory material before reaching our desired statements. For any subset A ⊆ X and x ∈ X , we define d i s t A , x = inf y ∈ A ρ y , x . (7)

The support of a finite positive measure μ, denoted by s u p p ( μ ) is the set of all x ∈ X such that μ(U) > 0 for all open subsets U containing x. Clearly, s u p p ( μ ) is a closed set. If σ is a non-zero signed measure and σ = σ ⁺ − σ ⁻ is the Jordan decomposition of σ then we define s u p p ( σ ) = s u p p ( σ + ) ∪ s u p p ( σ − ) . If f : X → R , we define ‖ f ‖ ∞ = sup x ∈ X | f x | .

The following lemma summarizes some important but easy properties of quantities Λ(f) and f ^# defined in Eqs. 4, 5 respectively.

Lemma 1

(a) For t, u ∈ [0, ∞), Λ f t ≤ u ⇒ t ≤ f # u , u ≤ Λ f t ⇒ f # u ≤ t . (8)

(b) If ϵ > 0, f , g : X → R , and ‖f − g‖ _∞ ≤ ϵ , then sup _u∈[0,∞)|f ^# (u) − g ^# (u)| ≤ ϵ .

Our goal is to investigate sufficient conditions on two measures in P δ so that kdiff can distinguish if the foreground components of the measures are the same. For this purpose, we introduce some further notation, where we suppress the mention of certain quantities for brevity. Let μ j = δ μ j , F + ( 1 − δ ) μ j , B ∈ P δ , j = 1, 2, and S F = s u p p ( μ 1 , F ) ∪ s u p p ( μ 2 , F ) , and we define S c = X / S . We define for η, θ > 0, S B μ 1 , μ 2 ; η = z ∈ X : T μ 1 , B − μ 2 , B z < η , ϕ B η = ν * S B μ 1 , μ 2 ; η S F μ 1 , μ 2 ; η = z ∈ X : T μ 1 , F − μ 2 , F z < η , ϕ F η = ν * S F μ 1 , μ 2 ; η G μ 1 , μ 2 ; θ , η = S F c θ ∩ S B η ∪ S B c θ ∩ S F η , ψ θ , η = ν * G μ 1 , μ 2 ; θ , η (9)

Theorem 2

Let δ ∈ ( 0 , 1 2 ) , μ j = δ μ j , F + ( 1 − δ ) μ j , B ∈ P δ ( j = 1,2 ) .

a) If η > 0 and μ _1,F = μ _2,F then for any α ≤ ϕ _B (η), we have kdiff (μ ₁, μ ₂; α) ≤ (1 − δ)η .

Proof

To prove part (a), we observe that since μ _1,F = μ _2,F , T(μ ₁ − μ ₂) (z) = (1 − δ) ⋅ T(μ _1,B − μ _2,B ) (z) for all z ∈ X . By definition (9), S B μ 1 , μ 2 ; η = z ∈ X : T μ 1 , B − μ 2 , B z < η = z ∈ X : T μ 1 − μ 2 z < 1 − δ η .

Therefore, ϕ _B (η) ≤ Λ(T (μ ₁ − μ ₂)) ((1 − δ)η). In view of (8), this proves part (a).

To prove part (b), we will write θ = 3 1 − δ δ η .

Our hypothesis that ϕ _F (θ) < 1 means that μ _1,F ≠ μ _2,F and S F c ( θ ) is nonempty. For all z ∈ S F c ( θ ) ∩ S B ( η ) , T μ 1 − μ 2 z ≥ δ | U μ 1 , F − μ 2 , F z | − 1 − δ | U μ 1 , B − μ 2 , B z | > δ θ − 1 − δ η ≥ 2 1 − δ η .

Moreover, for z ∈ S B c ( θ ) ∩ S F ( η ) , we also know that T μ 1 − μ 2 z ≥ 1 − δ | U μ 1 , B − μ 2 , B z | − δ | U μ 1 , F − μ 2 , F z | ≥ 1 − δ θ − δ η ≥ 3 1 − δ 2 − δ 2 δ η .

Note that because δ < 1 2 , we have 3 ( 1 − δ ) 2 − δ 2 δ > 2 ( 1 − δ ) . So, this means that z ∈ X : T μ 1 − μ 2 z < 2 1 − δ η ⊂ z ∈ X : z ∉ G θ , η ;

This means that Λ(T (μ ₁ − μ ₂)) (2 (1 − δ)η) ≤ 1 − ψ(θ, η). Since α ≥ 1 − ψ(θ, η), this estimate together with (8) leads to the conclusion in part (b).

We wish to comment on the practicality of the constants S F ( μ 1 , μ 2 ; η ) , S B ( μ 1 , μ 2 ; η ) and G ( μ 1 , μ 2 ; θ , η ) . We consider this with the simple setting where K is a compactly supported localized kernel (e.g., indicator function of an ϵ-ball) in order to avoid the discussion of tails. We define the well-separated setting as the setting where d i s t ( s u p p ( μ 1 , B ) , s u p p ( μ 2 , B ) ) > ϵ and d i s t ( s u p p ( μ 1 , B ) , s u p p ( μ i , F ) ) > ϵ . For part (b), we’ll also use d i s t ( s u p p ( μ 1 , F ) , s u p p ( μ 2 , F ) ) > ϵ and all four measures are sufficiently concentrated, i.e., μ i , F z ∈ X : T μ i , F z ≥ θ ≥ 1 − ξ  and  μ i , B z ∈ X : T μ i , B z ≥ θ ≥ 1 − ξ .

We consider the results of Theorem 2 in the well-separated setting with ν * = 1 2 ( μ 1 + μ 2 ) :

a) S B ( μ 1 , μ 2 ; η ) measures how much the backgrounds overlap with one another. In this setting, ϕ _B (η) ≥ δ for any η > 0. This is because T (μ _1,B − μ _2,B ) (z) = 0 for all z ∈ s u p p ( μ i , F ) , and thus z ∈ S B ( μ 1 , μ 2 ; η ) . Since ν * ( s u p p ( μ 1 , F ) ∪ s u p p ( μ 2 , F ) ) = δ , this lower bounds ϕ _B (η). This means for any α < δ, kdiff (μ ₁, μ ₂; α) = 0.

Thus we can choose η, θ as large as possible to satisfy the assumptions, and even then for very small quantiles α > ξ we kdiff (μ ₁, μ ₂; α) > 2 (1 − δ)η.

These above descriptions clarify the theorem in the simplest setting. When the foreground distributions are small but concentrated, and far from the separate backgrounds, then the hypothesis of μ 1 , F = ? μ 2 , F can be easily distinguished with kdiff for almost all α < δ.

In practice, of course, we need to estimate kidff (μ ₁, μ ₂; α) from samples taken from μ ₁ and μ ₂. In turn, this necessitates an estimation of the witness function of probability measure from samples from this probability. We need to do this separately for μ ₁ and μ ₂, but it is convenient to formulate the result for a generic probability measure μ. To estimate the error in the resulting approximation, we need to stipulate some further conditions enumerated below. We will denote by S * = s u p p ( μ ) .

Essential compact support For any t > 0, there exists R(t) > 0 such that K x , y ≤ t , x , y ∈ X , ρ x , y ≥ R t . (10)

Covering property For t > 0, let B ( S * , t ) = { z ∈ X : d i s t ( S * , z ) ≤ R ( t ) } . There exist A, β > 0 such that for any t > 0, the set B ( S , t ) is contained in the union of at most At ^−β balls of radius ≤ t.

Lipschitz condition We have max x , y ∈ X × X K x , y + max x , x ′ , y , y ′ ∈ X | K x , y − K x ′ , y ′ | ρ x , x ′ + ρ y , y ′ ≤ 1 . (11)

Then Höffding’s inequality leads to the following theorem.

Theorem 3

Let ϵ > 0, M ≥ 2 be an integer, and μ be any probability measure on X and {y ₁, … , y _M } be i.i.d. samples from μ . Then with μ -probability ≥ 1 − ϵ , we have U μ ◦ − 1 M ∑ j = 1 M K ◦ , y j ∞ ≤ 2 log 4 β + 1 A / ϵ M 1 / 2 . (12)

The proof of Theorem 3 mirrors the results for the witness function in [14].

To illustrate the benefit of the above theory, we recall the MMD distance measure between two probability measures μ ₁ and μ ₂ defined by MMD 2 μ 1 , μ 2 = ∫ X ∫ X K x , y d μ 1 x − d μ 2 x d μ 1 y − d μ 2 y . (13)

When μ 1 , μ 2 ∈ P δ and the foreground components μ _1,F = μ _2,F then μ ₁ − μ ₂ = (1 − δ) (μ _1,B − μ _2,B ) and MMD 2 μ 1 , μ 2 = 1 − δ 2 MMD 2 μ 1 , B , μ 2 , B . (14)

Since K is a positive definite kernel, it is thus impossible for MMD² (μ ₁, μ ₂) = 0 unless μ _1,B = μ _2,B . One of the motivations for our construction is to devise a test statistic that can be arbitrarily small even if μ _1,B ≠ μ _2,B .

The results derived above provide certain insights regarding when it is possible to perform tasks such as clustering and classification of data using distance measures such as kdiff and MMD based on the characteristics of their foreground and background distributions. The results in Theorem 2 show that, provided the backgrounds are sufficiently separated, the kdiff statistic will be significantly smaller when μ _1,F = μ _2,F than when μ _1,F and μ _2,F are separated.

This enables kdiff to perform accurate discrimination i.e. data belonging to the same class will be clustered correctly in this case. On the other hand, it is clear that even if μ _1,F = μ _2,F , MMD² (μ ₁, μ ₂) will still be highly dependent on the backgrounds. In this paper we consider the case where data instances belonging to the same class have the same foreground but different background distributions. In such situations using synthetic and real life examples we demonstrate the comparative performance and effectiveness of kdiff for clustering tasks versus other distance measures including MMD.

As a final note, we wish to mention the relationship between MMD and kdiff. It can be shown that MMD ² is the mean of the witness function with respect to ν * = 1 2 ( μ 1 + μ 2 ) , M M D 2 ( μ 1 , μ 2 ) = E z ∼ ν * | U ( μ 1 − μ 2 ) ( z ) | 2 [11, 15]. This is compared to our results for kdiff, or in particular kdiff². Note that computing k d i ff ( μ 1 , μ 2 ; α ) 2 is equivalent to computing kdiff on the square of the witness function T (μ ₁ − μ ₂) (z) = |U (μ ₁ − μ ₂) (z)|², since quantiles depend only on the ordering of the underlying function. This means the statistic kdiff² is simply taking the quantile of the square of the witness function, rather than the mean as in MMD ².

Data Y _i for i = 1, 2, … , 1,000 are simulated using the model (15). To generate this the series W _i are constructed via an AR (5) model driven by i. i.d errors ∼ N ( 0,1 ) . The AR (5) coefficients are set to 0.5, 0.1, 0.1, 0.1, 0.1. Y i = μ + W i (15)

Following this we form a background dataset X B j by generating j = 1, 2, … , 21 realizations of this data where the mean μ _j for realization j is set as below: μ j = 100 ∗ j if  j ≥ 1 and  j ≤ 10 100 ∗ 10 − j if  j ≥ 11 and  j ≤ 20 0 if  j = 21

Next we generate a dataset X _F consisting of 2 foregrounds X _FA and X _FB which enable forming the 2 classes to be considered for k-medoids clustering as follows. For foreground X _FA data Y _i for i = 1, 2, … , 50 are simulated using the model (15). The series W _i is constructed via an AR (1) model driven by i. i.d errors ∼ N ( 0,1 ) . The AR (1) coefficient is set to 0.1 and μ = 10. For foreground X _FB data Y _i for i = 1, 2, … , 25 are simulated using the model (15). The series W _i is constructed via an AR (1) model driven by i. i.d errors ∼ N ( 0,1 ) . The AR (1) coefficient is set to 0.1 and μ = − 10. We then form the foreground dataset X F j by generating j = 1, 2, … , 21 realizations of this data as follows: X F j = X F A if  j mod  2   = 1 X F B if  j mod  2   = 0

Finally the dataset used for clustering Z _ij where i = 1, 2, … , 1,000 and j = 1, 2, … , 21 is formed by mixing backgrounds X _B and foregrounds X _F as follows: Z i j = ∑ i = 1 50 X F i j + ∑ i = 51 1000 X B i j if  j mod  2   = 1 ∑ i = 1 25 X F i j + ∑ i = 26 1000 X B i j if  j mod  2   = 0

The dataset Z _ij formed in this manner consists of two types of subregions (foregrounds) which define the two classes used for k-medoids clustering. We perform 50 random splits of the dataset Z _ij where each split consists of a training set of size 10 and a test set of size 11. The results for clustering are shown for the 4 distance measures in Table 1.

From the results it can be seen that both kdiff and MPdist produce the best clustering performance with 0 errors for this dataset. This is attributed to the fact that the subregions of interest are well defined for both classes and using suitable values of parameters determined from training it is possible to accurately cluster all the time series data into two separate groups. On the other hand the performance of MMD is inferior to both kdiff and MPdist because the backgrounds are well separated with different mean values for time series within and across the two classes. This results in time series even belonging to the same class to be placed in separate clusters when MMD is used as a distance measure. Similarly dtwd suffers from poor performance as this distance measure tends to place time series with smaller separation between the mean background values in the same cluster. However these may have distinct values for the foregrounds i.e., they can in general belong to different classes and as a result this causes errors during clustering.

Noise robustness We explore the performance of the distance measures by considering noisy foregrounds. For foreground X _FA data Y _i for i = 1, 2, … , 50 are simulated using the model (15). The series W _i is constructed via an AR (1) model driven by i. i.d errors ∼ N ( 0,100 ) . The AR (1) coefficient is set to 0.1 and μ = 10. For foreground X _FB data Y _i for i = 1, 2, … , 25 are simulated using the model (15). The series W _i is constructed via an AR (1) model driven by i. i.d errors ∼ N ( 0,1 ) . The AR (1) coefficient is set to 0.1 and μ = − 10. Following this the foreground datasets X F j and the dataset used for clustering Z _ij where i = 1, 2, … , 1,000 and j = 1, 2, … , 21 are formed in the same manner as described earlier. We show example time series realizations for τ = 1 and 10 in Figures 1, 2 respectively. Each figure contains plots of two time series with mean = − 10, 10 as per the construction of foreground X _FB for the original and noisy case and show the relative separation between the realizations.

The results for clustering using these noisy foregrounds are shown in Table 2. The data shows empirically that as the noise level of the foreground increases kdiff is more resilient and performs better than MPdist. This is because after constructing sliding window based embeddings over the original data, MPdist is computed using Euclidean metric based cross-similarities between the embeddings whereas kdiff is estimated using kernel based self and cross similarities over the embeddings.

We generate a 3d multivariate background dataset s _B as follows. Data Y a i for i = 1, 2, … , 1,000 are simulated using the model (15). To generate this the series W _i is constructed via an AR (1) model driven by i. i.d errors ∼ N ( 0,1 ) . The AR (1) coefficient is set to 0.1. Similarly data Y b i for i = 1, 2, … , 1,000 are simulated using the model (15). To generate this the series W _i is constructed via an AR (1) model driven by i. i.d errors ∼ N ( 0,1 ) . The AR (1) coefficient is set to 0.1.

Following the generation of W _i values for the data Y = (Y _a , Y _b ) we form a background dataset X B j in this 2d space by generating j = 1, 2, … , 21 realizations of this data Y where the mean μ for realization j of each pair is set as below: μ = 100 ∗ j if  j ≥ 1 and  j ≤ 10 100 ∗ 10 − j if  j ≥ 11 and  j ≤ 20 0 j = 21

Our next step involves transforming these 21 instances of the 2d backgrounds into a 3d spherical surface of radius 1 as described in the following steps. We first map each series Y _a and Y _b linearly into the region [0, π/2]. The corresponding mapped series are denoted as Y a s and Y b s respectively. To ensure that the backgrounds are clearly separated we divide the region [0, π/2] into 21 nonoverlapping partitions for this linear mapping. The final background dataset s _B = {s _a , s _b , s _c } is derived using : s a = sin Y b s ∗ cos Y a s s b = sin Y b s ∗ sin Y a s s c = cos Y b s (16)

Next we generate a 3d foreground dataset s _F consisting of 2 foregrounds s _FA and s _FB which will enable forming the 2 classes to be considered for k-medoids clustering as follows. Data Y a i for i = 1, 2, … , 50 are simulated using the model (15). To generate this the series W _i is constructed via an AR (1) model driven by i. i.d errors ∼ N ( 0,1 ) . The AR (1) coefficient is set to 0.1 and μ = 10. Similarly data Y b i for i = 1, 2, … , 50 are simulated using the model (15). To generate this the series W _i is constructed via an AR (1) model driven by i. i.d errors ∼ N ( 0,1 ) . The AR (1) coefficient is set to 0.1 and μ = 10. we linearly map the original 2d data (Y _a , Y _b ) into the region [π/2, 5π/8] as ( Y a s , Y b s ) and then perform the mapping as given in Eq. 16 to form the foreground s _FA . The foreground s _FB is generated in a similar manner except that μ = − 10 and the 2d series is linearly mapped to the region [3π/4, 7π/8]. We form the foreground dataset s F j by generating j = 1, 2, … , 21 realizations of this data as follows: s F j = s F A if  j mod  2   = 1 s F B if  j mod  2   = 0

Finally the dataset used for clustering Z _ij where i = 1, 2, … , 1,000 and j = 1, 2, … , 21 is formed by mixing backgrounds s _B and foregrounds s _F as follows: Z i j = ∑ i = 1 50 s F i j + ∑ i = 51 1000 s B i j if  j mod  2   = 1 ∑ i = 1 25 s F i j + ∑ i = 26 1000 s B i j if  j mod  2   = 0

The dataset Z _ij formed in this manner consists of two types of subregions (foregrounds) which define the two classes used for k-medoids clustering. An illustration of the data on such a spherical surface with 5 backgrounds and 2 foregrounds is shown in Figure 3.

We perform 50 random splits of the dataset Z _ij where each split consists of a training set of size 10 and a test set of size 11. The results for clustering are shown for the 4 distance measures in Table 3.

From the results it can be seen that kdiff produces the best clustering performance with 0 errors for this dataset. This is attributed to the fact that the subregions of interest are well defined for both classes and using suitable values of parameters determined from training it is possible to accurately cluster all the time series data into two separate groups. On the other hand the performance of MMD is inferior to kdiff because the backgrounds are well separated with different mean values for time series within and across the two classes. This results in time series even belonging to the same class to be placed in separate clusters when MMD is used as a distance measure. Similarly dtwd suffers from poor performance as this distance measure tends to put time series with smaller separation between the mean background values in the same cluster. However these may have distinct values for the foregrounds i.e. they can in general belong to different classes and as a result this causes errors during clustering. For this dataset the performance of MPdist is inferior to kdiff even though the former can find matching sub-regions with zero errors in the case of univariate time series. This difference is attributed to the nature of the spherical region over which the sub-region matching is done where the 1-nearest neighbor strategy employed by MPdist using Euclidean metrics to construct the distance distribution. In case of spherical surfaces it is necessary to use appropriate geodesic distances for nearest neighbor search as discussed in ([18]). This issue is resolved in kdiff which can find the matching subregion over a non Euclidean region which in this case is a spherical surface thereby giving the most accurate clustering results for this dataset.

The MNIST-M dataset used in [19, 20] was selected as a real-life example to demonstrate the differences in clustering performance using the four distance measures kdiff, MMD, MPdist and dtwd. The MNIST-M dataset consists of MNIST digits [21] which are difference blended over patches selected from the BSDS500 database of color photos [22]. In our experiments where we consider k-medoid clustering over k = 2 classes we select 10 instances each of the MNIST digits 0 and 1 to be blended with a selection of background images to form our dataset MNIST-M-1. Since BSDS500 is a dataset of color images the components of this dataset are random fields whose dimensions are 28 × 28 × 3. We form our final dataset for clustering consisting of random fields with dimensions 28 × 28 by averaging over all three channels. Examples of individual zero and one digits on different backgrounds for all three channels of MNIST-M-1 are shown in Figures 4–7.

We perform 50 random splits of the dataset where each split consists of a training set of size 10 and a test set of size 10. The results for clustering are shown for the distance measures in Table 4.

From the results it can be seen that for MNIST-M-1 MPdist somewhat outperforms our proposed distance measure kdiff however the latter is superior to both MMD and dtwd. Since in general the background statistics of the MNIST-M images are different, two images belonging to the same class can be placed in separate clusters when MMD is used as a distance measure and this causes MMD to underperform versus kdiff. Similarly dtwd suffers from poor performance as this distance measure tends to put images with smaller separation between the mean background values in the same cluster. However these may have distinct values for the foregrounds i.e. they can in general belong to different classes and as a result this causes errors during clustering.

Noise robustness Following the discussion in Section 4.1 we explore the performance of the distance measures by considering a selection of noisy backgrounds from the BSDS500 database over which the same 10 instances of the MNIST digits 0 and 1 are blended to form a second version of our dataset called MNIST-M-2. Similar to the earlier case we form our final dataset for clustering consisting of random fields with dimensions 28 × 28 by averaging over all three channels of the color image. Examples of individual zero and one digits on different backgrounds for a single channel are shown in Figures 8–11. Note that these correspond to the same MNIST digits shown in Figures 4–7 however are blended with different backgrounds which have been chosen such that the distinguishability of the two classes is reduced.

We use the Kolmogorov-Smirnov (KS) test statistic to characterize the differences between the backgrounds (BSDS500 images) and the foregrounds (MNIST digits 0 and 1) as below:

• The mean KS statistic between the distribution of the pixels where a digit 0 is present and the distribution of the pixel values which make up the background (KS-bg-fg-0)

The KS values shown in Table 5 confirm our visual intuition that the distinguishability of the foreground (MNIST 0 and 1 digits) and the background is less for MNIST-M-2 as compared to MNIST-M-1. Additionally it can be seen that for the noisier dataset MNIST-M-2 the separation between the background distribution of pixels is less than that of MNIST-M-1.

We perform 50 random splits of the dataset Z where each split consists of a training set of size 10 and a test set of size 10. The results for clustering are shown for the distance measures in Table 6.

From the results it can be seen that for this noisy dataset the clustering accuracy results for all four distance measures are lower as expected, however kdiff slightly outperforms MPdist. As discussed in Section 4.1 this can be attributed to the fact that in such cases with a lower signal to noise ratio between the foreground and the background kdiff which is estimated using kernel based self and cross similarities over the embeddings can outperform MPdist which is computed using only Euclidean metric based cross-similarities over the embeddings. The expected noise characterizaion is confirmed by our KS statistic values of KS-bg-fg-0 and KS-bg-fg-1 in Table 6. Moreover the lower values of the KS statistic value KS-bg for MNIST-M-2 compared to MNIST-M-1 manifest in similar clustering performances of MMD and kdiff for MNIST-M-2 in contrast with the trends for MNIST-M-1.

Additional comments: For kdiff we used L = SL ∗ SL windows for capturing the image sub-regions leading to (n − SL + 1)² embeddings which were subsequently “flattened” to form subsequences of size L = SL ² over which kdiff was estimated using a one dimensional Gaussian kernel. This process can be augmented by estimating kdiff with two dimensional anisotropic Gaussian kernels to improve performance. However this augmented method of kdiff estimation using a higher dimensional kernel with more parameters will significantly increase the computation time and implementation complexity. Note that in the case of MPdist flattening the subregion is not as much of an issue since it does not use kernel based estimations which need accurate bandwidths.