In Curtains conditional invertible neural networks (cINNs) (Ardizzone et al., 2019a,b) are employed to transform data points from an input distribution to those from the target distribution. The transformation is conditioned on a function f of the resonant feature m_JJ of the input and target data points. Unlike flows (Rezende and Mohamed, 2016; Kobyzev et al., 2021), which use the exact maximum likelihood of transforming data to a desired distribution, usually a multivariate normal distribution, we use an optimal transport loss to train the network to transform data between the two desired distributions.

As the cINN can be used in both directions, the inputs to the conditional function are referred to as the lower and higher values mJJlow and mJJhi. In the case of a forward pass through the network, mJJlow are the true values of the input data, with mJJhi the target values, and vice versa in the case of an inverse pass. Furthermore, instead of training the cINN in only the forward direction, we iterate between both the forward and inverse directions to ensure better closure between the output and target distributions and to prevent a bias toward transformations in one direction.

Several different network architectures for the transformer were studied in the development of Curtains. The transformers presented in this paper are built on the invertible transformations introduced in Durkan et al. (2019) which use rational-quadratic (RQ) splines, which are found to be very expressive and easy to train. The conditioning function f is chosen to be

The features which are to be transformed determine the input and target dimensions of the Curtains transformer.

To train the network, batches of data are drawn from the low-mass side-band SB1 and the high-mass side-band SB2. The data from SB1 is first fed through the network in a forward pass, conditioned using mJJlow and mJJhi, with target values for each event assigned by randomly pairing the masses drawn from each side-band. The loss between the transformed data and target data is calculated using the sinkhorn divergence (Cuturi, 2013) across the whole batch, in order to measure the distance between the distributions of the two sets of data. The gradient of this loss is used to update the network weights. In the next batch the data from SB2 is fed through the network in an inverse pass and the same procedure is performed. This alternating procedure is repeated throughout the training. A schematic overview of the Curtains transformer model is shown in Figure 1.

With this training procedure the optimal transport function is not exactly derived as the conditional information is only implicit and a transformed event will not necessarily be paired to the event with the mass to which it was mapped in the loss calculation. However, after training the network we observe that the learned transformation is a good approximation of the true optimal transformation.

In order to improve the closure of the transformed data to regions other than the side-bands, an additional training step is performed. After an epoch of training the network between SB1 and SB2, each side-band itself is split into two equal width sub side-bands. The network is then trained for an epoch of each intra side-band training, following the same procedure as for the inter side-band training. Although not necessary for the Curtains method, this extra step is performed in order to extend the range of values of the conditional information used to train the network. Instead of having a minimum value of f(mJJlow,mJJhi) equal to the width of the signal region separating SB1 and SB2, its minimum values is now zero. This ensures that the conditioning variables used to map data to the signal region always lie in the distribution of values used during training.

The Curtains transformer is trained for 1,000 epochs with a batch size of 256 using the Adam optimiser (Kingma and Ba, 2017). A cosine annealing learning rate schedule is used with an initial learning rate of 10⁻⁴. A typical training time of 6 h using an NVIDIA^® 3080 RTX GPU is required for a central window encompassing 10⁵ samples across the two side-bands.

The Curtains transformers are trained separately for each step in the sliding window, using all the available data in the side-bands. In order to construct a background template in another region, all the data from SB1 and SB2 are transformed in either a forward or inverse pass to mass values sampled from the target window. To create the background template in the signal region, the data from SB1 (SB2) are transformed to values of m_JJ corresponding to the signal region in a forward (inverse) pass with the Curtains transformer. These two transformed datasets are combined to create the background template in the signal region.

In the case of validating the Curtains transformer, the side-band data can be transformed to a target window with the same width as the signal region but going in the opposite direction in m_JJ, defining outer-band regions for SB1 (OB1) and SB2 (OB2). These regions can be used to validate and tune the Curtains method in a real world setting. The five bands of one sliding window are illustrated in Figure 2, with the depicted signal region centred on the invariant mass of the injected signal. In the studies presented in this paper the width of the side-bands and validation regions is set to 200 GeV by default, unless otherwise specified. To increase the statistics of the constructed datasets the transformer can be applied many times to the same data with different m_JJ target values in each pass.

The hyperparameters and architecture of the Curtains transformer were optimised in a grid search by measuring the agreement between data transformed into the two outer-band regions from the two side-bands for one step of the sliding window without any doping of signal events. The agreement is measured by training a classifier to separate the two datasets and ensuring the Receiver Operator Characteristic (ROC) curve had a linear response with an area under the curve close to 0.5, which suggests the network was unable to differentiate between real and transformed data in this region. The optimal CURTAINs transformer is made up of eight stacked RQ spline coupling layers. Each coupling layer is constructed from three residual blocks each of two hidden layers of 32 nodes with LEAKY RELU activations, resulting in an output spline with four bins. The n-flows package (Durkan et al., 2020) is used to implement the network architecture in Pytorch 1.8.0 (Paszke et al., 2019). These settings are then used to train all Curtains transformers for each step of the sliding window, and for all doping levels.

In order to sample target values for the Curtains transformer and not be biased to the presence of any excess of events in the signal region, the distribution of the resonant feature in the signal region needs to be extrapolated from the side-band data. Here we model the QCD dijet background with the functional form

where z=mJJ/s with the centre of mass energy of the collision s=13 TeV. The parameters p₁, p₂, and p₃ are obtained from an unbinned fit to the side-band data in using the zfit package (Eschle et al., 2020). This Ansatz has been used previously in analyses performed at the LHC (ATLAS Collaboration, 2016) and is similar to that used in more recent searches with the omission of the last free parameter (CMS Collaboration, 2018; ATLAS Collaboration, 2020). Once fit to the side-band data, the learned parameters are used in the PDF from which to sample target m_JJ values for the transformer.

Once the background data has been transformed into the signal region from the side-bands, it is possible to use them as the background template to test for the presence of signal in the data from the region. There are several approaches which could be used for anomaly detection with the data transformed with the Curtains method, however in this paper we will focus on the CWoLa classifier, as applied also in Collins et al. (2019), Benkendorfer et al. (2021), and Hallin et al. (2022) on this dataset.

For a CWoLa classifier, it can be shown that the performance of a classifier trained on two sets of data, each containing a different mixture of signal and background data will result in the optimal classifier trained on pure sets of signal and background data. Here, we assume our transformed data represents a sample of pure background events, and test the hypothesis that in our signal region data there is a mixture of signal and background data. In the presence of signal events in the signal region, the classifier will be able to separate the signal region data from the background template, with the true signal events having higher classification scores than the true background data. By applying a cut on the classifier output to reject a given fraction of the background, calculated from the scores of the background template, the significance of the signal events can be enhanced.

In cases where there is signal contamination in at least one of the side-bands of the sliding window, the background template constructed with Curtains will also contain a non-zero signal to background fraction. With the assumption that the signal is localised, and the bin widths are not too small, the relative fraction of signal in the signal region will be different from the background template. As such, the CWoLa method will still be able to approach the performance of the ideal classifier. The background template provided by Curtains will have a lower signal to background ratio than the signal region in at least one step of the sliding window, and in this bin an excess can be expected.

In the event of the signal being fully localised within a side-band, this will result in the opposite labels being used in the training of the CWoLa classifier with regards to which dataset contains the higher fraction of signal. After applying a cut on the classifier a slight reduction in events with respect to the prediction could therefore be expected. However, in practise we observe no significant deviation with the dataset under consideration.

The values used as acceptance thresholds on the output of the classifier are independently determined for each classifier in the signal regions across all sliding windows and levels of doping. These cuts are used to enhance the sensitivity to the presence of signal data in each window of the fit. The amount of data which remains after the cut can be compared to the expected background, determined by taking the total number of data in each signal region multiplied by the background rejection factor. In the presence of a signal, a significant excess of data will be observed above the expected background.

A further test of the performance when using the CURTAINs method is to compare against three benchmark classifiers. The first is a fully supervised classifier, trained with knowledge of which events were from the signal process and which were QCD background. Two further classifiers, the idealised classifiers, are trained in the same manner as with the CURTAINs background template, except that the background template comprises true background data from the signal region itself.

Both the supervised and idealised classifiers are only trained for the window in which the signal region is aligned with the peak of the signal data. The supervised classifier provides an upper bound on the achievable performance on the dataset. The idealised classifier sets the target level of performance which can be achieved with a perfect background template, and can be used to validate the performance of Curtains for use in anomaly detection.

All the classifiers for all signal regions and all levels of signal doping share the same architecture and hyperparameters. The classifiers used in this paper have been chosen as they are robust to changes in datasets and initial conditions, in particular when using k-fold training and low training statistics. The classifiers are constructed from multilayer perceptrons with three hidden layers with 32 nodes and RELU activations. The classifiers are trained for 20 epochs using the Adam optimiser with a batch size of 128, and an initial learning rate of 0.001 which anneals to zero following a cosine curve over 20 epochs.

The first test of performance in Curtains is to demonstrate that the transformation learned between the two side-bands is accurate, and further to determine whether the learned transformation can extrapolate well to the validation regions. As Monte Carlo simulation is being used for the studies, we can control the composition of the samples in the studies. The performance of the approach is evaluated using a sample containing only background data, as well as various levels of signal doping. The same model configuration is used for all samples, and the sliding window is chosen such that it is centred on the true signal peak with a signal region width of 200 GeV.

The input features and their correlations for the input, target and transformed data distributions are shown in Figure 3 for the two side-bands trained in the case of no signal, and in Figure 4 for the two validation regions. As can be seen, the transformed data distributions are well reproduced with the Curtains approach. The ability of Curtains to handle features which are strongly correlated with m_JJ can be seen from the agreement of the ΔR_JJ distributions between SB1 and SB2, which exhibit very different shapes.

In the case of no signal being present, we can also verify whether the background template constructed by transforming data from the side-bands with Curtains matches the target data in the signal region. The performance of the Curtains method can be seen in Figure 5, with the transformed data closely matching the data distributions and correlations.

To quantify the level of agreement between the transformed distributions and the target data, classifiers are trained to separate the two datasets, and the area under the ROC curve is measured. The level of agreement between the Curtains transformed data and target data can be seen for several levels of signal doping in Table 1. We can see that Curtains has very good agreement with the target distribution in all signal regions and in all cases is seen to be better than in the validation region. The reduced performance in OB1 and OB2 is a result of the transformer extrapolating outside of the trained sliding window. This demonstrates their ability to be used for validating the Curtains method and future classification architectures.

To demonstrate the performance of Curtains to produce a robust background template, the sliding window is centred on the resonant mass of the signal events, and the performance of the CWoLa classifier is compared against a background template produced using the Cathode method. The signal region width is set to 400 GeV to contain the majority of the signal events, resulting in 120,000 background events. The background template is produced with oversampling, with a total of nine times the number of expected events in the signal region. Two comparisons to Cathode can be performed, one using the same training windows as for Curtains, which we refer to as Cathode (local), and one using the full invariant mass distribution outside the signal region, as presented in Hallin et al. (2022), which we refer to as Cathode (full).

For reference, the methods are compared to a classifier trained using an idealised background template and to a fully supervised classifier. The idealised background template constructed using true background events from the signal region, and the supervised classifier is trained to separate the signal data from the background data using class labels. To construct the idealised background dataset we either use an equal number of background data as there are in the signal region (Eq-Idealised) to measure the performance assuming we had access to a perfect model of the background data, or the same number of data points as are produced with the Curtains and Cathode approaches (Over-Idealised), which should approach the best possible performance for models which can oversample.

The performance of the classifiers with the different methods are shown for the doped sample with 3,000 injected signal events (of which 2,214 are in the signal region) in Figure 6, comparing the background rejection as a function of signal efficiency and the significance improvement as a function of the background rejection. In order to maintain a fair comparison to the Eq-Idealised classifier, which requires true background data from the signal region for the background template, only half of the available data in the signal region is used for training with the k-fold strategy for all other approaches. The maximum significance improvement is shown for a wide range of doping levels in Figure 7. This metric is a good measure of performance for anomaly detection, rather than the area under the ROC curve, as it translates to the expected performance gain when applying an optimal cut on a classifier.

We can see that Curtains not only outperforms Cathode (local), but also approaches the performance of the Over-Idealised and supervised scenarios. When using the full range outside of the signal regions to train Cathode (full) the performance recovers and Curtains is only able to match the performance at high levels of background rejection, as seen in Figure 8. However, this demonstrates that Curtains is able to reach a higher level of performance when trained on lower numbers of events.

As it is not possible to know the location of the signal events when applying Curtains to data, the real test of the performance and robustness of the method is in the sliding window setting.

Both Curtains and Cathode (local and full) are used to generate the background templates in a sliding window scan in the range 3,000–4,600 GeV, with steps of 200 GeV and equal 200 GeV wide signal regions. Classifiers are trained to separate the signal region data from the background template, and cuts are applied to retain 20, 5, and 0.1% of the background events.

These scans are performed for several levels of signal doping and are shown in Figure 9 for the case where there is no signal present, and in Figure 10 for doped samples with 500, 667, 1,000, and 8,000 injected signal events. Each signal region is subdivided into two bins of equal width in m_JJ for the plot. The expected background is determined by multiplying the original yield of each bin by the chosen background retention factor.

In contrast to the CWoLa bump hunt approach introduced in Collins et al. (2019), which uses the classifier trained in the signal region to apply a cut on all events on the invariant mass spectrum before performing a traditional bump hunt, we treat each signal region as an independent region and do not apply the classifiers outside of the regions in which they are trained. This sliding window approach tests how the Curtains and Cathode approaches perform as the sideband windows used to train the networks as well as the signal region transition between the presence of signal, to signal in one of the sidebands as well as the case where there is perfect alignment of signal in the signal region. This approach does not test the ability of the trained classifiers to extrapolate outside of the values of invariant mass used to train them. Were they to be applied outside of the respective regions it is expected sculpting of the invariant mass distribution would occur after applying cuts on the classifier due to the strong correlation between ΔR_JJ and m_JJ.

As can be seen from the sliding window scans in Figures 9, 10, using Curtains and Cathode (full) we are able to correctly identify the location of the signal events for even reasonably low levels of signal. Where there are no or very few signal events the yields after each cut do not deviate too far from expected background. The corresponding significance of excesses seen in each bin for three cuts on background efficiency are shown in Figure 11. In the case where there is no signal injected into the sample, both Curtains and Cathode (full) have relatively low local excesses reaching a 4σ deviation only in one bin at the 1% background efficiency and not exceeding 3σ deviations for tighter cuts when considering only statistical uncertainties on the yields in each bin.

However, at lower levels of background rejection, significant local excesses over the expectation are observed. At 5% background efficiency a maximum local deviation is observed at 4σ for Cathode (full) and 5σ for Curtains. In the presence of signal both Curtains and Cathode (full) have observed excesses at the signal mass peak for each cut level for the lower levels of signal events, with Cathode (full) approaches results in a more prominent excess.

The Cathode (local) approach yields an excess across the whole spectrum in the absence of signal and for all levels of injected signal. However, it also finds an excess under the signal peak in the cases where signal is injected, which at higher levels of signal exceeds that found by Curtains.

In an analysis a systematic excess over the expectation calculated from the original yields per bin would not necessarily be problematic, as the expectation could be determined from a side band fit in m_JJ after applying the cut. Additionally, these values do not take any systematic uncertainties into account and only consider statistical uncertainties on the number of events passing each cut from the yields.

Although the ability to isolate the signal events when using Curtains in the window scan decreases at low numbers of signal events and signal purity, this is also seen for both idealised cases in Figure 7 and suggests this as rather an area where the classifier architecture and anomaly detection method need to be optimised. The performance of Curtains in this setting could also be further improved by optimising the binning used in the sliding window, and the number of subdivisions within each signal region.