NYPD launched the Stop-and-Frisk program for recording and analyzing police officer’s regular enforcement practice. In the Stop-and-Frisk program, there are generally three types of actors: 1) the official police, who can stop a person and check whether there are weapons or drugs carried by the suspect, with filling out a form recording the details; 2) the suspect, who is subjected to the stops; and 3) the environment,in which the stops occur, including location illustration and time records. After an individual is stopped, officers may conduct a frisk (i.e., a quick pat-down of the person’s outer clothing) if they reasonably suspect the individual is armed and dangerous; officers may additionally conduct a search if they have probable cause of criminal activity. An officer may decide to make an arrest or issue a summons, all of which is recorded on the UF-250 form. Responses are subsequently standardized, compiled and released annually to the public.

Table.1 shows the description of the NYCSF dataset. The NYCSF dataset contains a variety of heterogeneous information about all entities. Entities of each type include various information in the form of unstructured data, such as text, and structured data, such as geo-spatial, numerical, categorical, and ordinal data.

In this paper, we mainly discuss the bias introduced by subject races, hence we view race as the sensitive attributes. This form records various aspects of the stop, including demographic characteristics of the suspect, the time and location of the stop, the suspected crime and the rationale for the stop (e.g., whether the suspect was wearing clothing common in the commission of a crime). One notable limitation of this dataset is that no demographic or other identifying information is available about officers. The forms were filled out by hand and manually entered into an NYCSF database until 2017, when the forms became electronic. The NYCSF reports NYCSF data in two ways: a summary report released quarterly and a complete database released annually to the public.

For the NYCSF program, we collect features from characteristic information and environment information perspectives, and highlight the key aspects of the data on Table 1.

We first set up the notations. Consider we have a crime prediction system with a driver’s feature set x _s ∈ S and a police set x _p ∈ P in the current environment x _e ∈ E. Let x _s represent subject information, like driver profile and behavior information; x _p represents a corresponding police officer for each driver, and x _e represents environment information. We set x = [ x s , x p , x e ] ∈ R d to concatenate heterogeneous data, where d is the number of feature dimensions. Each driver has a binary label t ∈ {1, 0} to denote whether the driver is searched by the police or not. In the following, to keep consistency, we set t = 1 as treatment and t = 0 as control. The associated labels y ∈ {0, one} are from the observational label space Y to represent the crime results. Formally, the collected history recording data D _n can be noted as a set of triples { ( x i , t i , y i ) } 1 ⩽ i ⩽ | D n | generated from an unknown distribution p _u (x, t, y) over driver’s feature-treatment-crime label space X × T × Y. The goal of our crime prediction framework is to learn a parametric function f _θ : X × T → Y from the available historical recording D _n to minimize the following true risk: L ( f θ ) = E P u ( x , t , y ) [ δ ( f θ ( x , t ) , y ) ] , (1) where we denote δ(f _θ (x, t), y) as the error function of the predicted output score with f _θ (x _i , t _i ) with the ground-truth label y. P _u (x, t, y) denotes the ideal unbiased data distribution. To be specific, bias refers to selection bias here, which means the searching actions are based on the police judgement.

Since the true risk is not accessible, the learning is conducted on the historical recordings D _n ∼ P _n (x, t, y) by optimizing the following empirical risk: L ( f θ ) = 1 | D n | ∑ i = 1 | D n | δ ( f θ ( x i , t i ) , y i ) , (2)

Based on the law of large numbers, the PAC theory Auer et al. (1995) states that empirical risk minimization can approximate the real risk if we have sufficient training data. With training instances sampled from random trials, the large amount of collected data can approximate the real data distribution for the reason of missing at random Shpitser (2016). However, as we mentioned above, because of the police selective enforcement, selection bias is demonstrated in this stop-and-frisk program.

Data analysis has demonstrated that racial prejudice is a confounder in the NYCSF program, and causes the police selective enforcement. Motivated by it, our goal is to approximate an ideal while unknown distribution p _u given observational data p _n . To deal with it, a general method is to training sample and obtain a re-weighted empirical risk function: L ̂ ( f θ | w ) = 1 | D n | ∑ i = 1 | D n | w i ⋅ δ ( f θ ( x i , t i ) , y i ) , (6) where the weighting parameter w _i is properly specified, i.e., w i = p u ( x i , t i , y i ) p n ( x i , y i , t i ) , measuring the discrepancy between selection bias eliminating data and observational data distribution. Such empirical risk is an unbiased estimation of the true risk.

As we mentioned above, selection bias arises from the fact that we only have one observational data, e.g., the polices stop and search the current data, while we known its counterfactual outcome, ‘what will the crime result be if the current driver is not searched by the police.’ To be general, we denote the counterfactual distribution as p _c (x, t, y(t)). It is intuitive that the ideal data distribution is the combination of observational p _n and counterfactual data p _c , i.e., p _u = p _n + p _c . Since we make the assumption that whether the driver is searched or not, his crime results should not be changed. Consequently, we make the corresponding counterfactual definition:

Definition (driver-based Counterfactual outcome.) Given driver’s feature x, his criminal label y, and treatment t. We assume his potential outcome as y(t), which denotes the expected criminal label with treatment t enforced. Concretely, we assume the label should not be changed, i.e., y(t) = y.

Assumption

(positivity). Given driver’s feature x, we assume the propensity score p(t|x) > 0. That is every driver has a positive probability to be searched by polices. This assumption is also consistent with the overlap assumption in causal inference.

In the following, we introduce how to model the counterfactual distribution p _c based on the observational data distribution p _n .

As we mentioned above, the re-weighted empirical risk function is an unbiased estimation of the true risk which approximates an ideal but unknown distribution, i.e., observational distribution p _n and counterfactual distribution p _c . To better estimate the weighting score, we estimate the counterfactual outcome distribution in this section, and connect its formulation with re-weighting method.

In a realistic world, we only observe y for either t = 1 or t = 0, and the corresponding counterfactual outcome is never observed, which connects causal inference with a missing data mechanism. With re-weighting techniques, the criminal results distribution in both observational and counterfactual world can be represented as : p α ( Y ( t ) , T | X ) = p ( Y ( t ) | T = t , X ) ⋅ p ( T = t | X ) ⋅ p α ( T | Y ( t ) , X ) p α ( T = t | Y ( t ) , X ) , (7) where p α ( T | Y ( t ) , X ) p α ( T = t | Y ( t ) , X ) counters the missing ratio in the observational distribution p _n . To be specific, we set T as t, which makes intervention equal to observational treatment. In this case, Eq. (7) can be represented as: p α ( Y ( t ) , T = t | X ) = p ( Y ( t ) | T = t , X ) ⋅ p ( T = t | X ) ⋅ p α ( T = t | Y ( t ) , X ) p α ( T = t | Y ( t ) , X ) = p ( Y ( t ) , T = t | X ) , (8)

In which p _α (Y(t), T|X) can be estimated from observational data. In this way, empirical risk would be equivalent to the true risk. To connect Eq. (7) with a counterfactual outcome, we set T = 1 − t and make the intervention to be the opposite action in the realistic world. p α ( Y ( t ) , T = 1 − t | X ) = p ( Y ( t ) , T = t | X ) ⋅ p ( T = t | X ) ⋅ p α ( T = 1 − t | Y ( t ) , X ) p α ( T = t | Y ( t ) , X ) . (9)

Specifically, we set t = 1 in Eq. (9) and p _α (Y(1), T = 0|X) represents the joint distribution that the driver is not searched by the police if he were searched. Since p(Y(t), T = t|X) and p(T = t|X) can be estimated from observational data, our goal on generate counterfactual outcome is to approximate p α ( T = t | Y ( t ) , X ) p α ( T = 1 − t | Y ( t ) , X ) including unknown factors Y(t). To ensure simplicity, we replace the term p α ( T = t | Y ( t ) , X ) p α ( T = 1 − t | Y ( t ) , X ) with a learnable function of g _α (T|X, Y(t)). Reminding our goal in Eq. (6), we minimize the real learned risk L ̂ : J ( θ , α ) ≔ min L ̂ ( θ , α ) = min 1 | D n | ∑ i = 1 | D n | g α ( T | X , Y ( t ) ) ⋅ δ ( f θ ( x i , t i ) , y i ) . (10)

Since we do not have ground truth for g _α (T|X, Y(t)) as supervised signals. To make Eq.(10) tractable, using definition 1, i.e., y(t) = y, we approximate g _α (T|X, Y(t)) to g _α (T|X, Y), which is a variant of inverse propensity score Austin (2011).

The NYCSF program can be taken as an example to illustrate the implication of g _α (T|X, Y). There are two dilemmas to understand selective enforcement p _α (T|X), where T ∈ {0, 1}. On the one hand, it is desirable to balance the distributions between the treated and the controlled groups, which can satisfy the unconfounded assumption in causal inference Hernán and Robins (2010): y ( 1 ) , y ( 0 ) ⫫ t | x . (11)

Given the latent feature representation h _i , the representation balancing methods design a distance measurement metric, e.g., f-divergence, to minimize the representation between P(h|x; t = 1) and P(h|x; t = 0). As a consequence, the outcome is independent with treatment given the input data.

On the other hand, in the NYCSF platform, data distribution in the treated and control group should be different: criminal drivers should have a higher probability to be stopped and searched by police. In this way, the data distribution should be indicative to its treatment prediction, i.e., P(h|x; t = 1) and P(h|x; t = 0) should contain distinguish feature representation and should not be similar with each other.

With this in mind, we assume g _α (T|X, Y) holding the following two properties:

• Selective Enforcement Equality. As a adjustment weight, g _α (T|X, Y) is expected to balance treatment groups with control groups w.r.t, the distributions of confounder representations, e.g., race prejudge. Hence the crime results are independent with the selective enforcement.

Since the aforementioned considerations contradict each other, we introduce an adversarial re-weight method for a better risk minimization.

Concretely, we introduce an auxiliary g _α to both approximate selection-bias eliminated data distribution and make the learned distribution distinguishable to its treatment. There are many variants of generative models that meet the two requirements Creswell et al. (2018). Inspired by existing works Xu et al. (2020); Hashimoto et al. (2018), we design a min-max game to solve the above problem. Here we regard the prediction model as a player and the weighting function as an adversary for simplicity. We can formulate our fairness objective as: J ( θ f , θ y , α t ) ≔ min θ f , θ y max α t L ̂ ( θ f , θ y , α t ) = min θ f , θ y max α t E x i ∼ D n g α ( f θ ( x ) , y ; α ) ⋅ δ ( f θ ( x i , t i ) , y i ) = min θ f , θ y max α t L ̂ ( θ f , θ y ) / L ̂ ( θ f , α t ) . (12)

To derive a concrete algorithm we need to specify how the players and adversary training parameters θ _f , θ _y , and α _t . At the saddle point, the feature mapping parameters θ _f both minimizes crime classification loss as well as maximizes treatment prediction loss. For parameters θ _y and α _t , they both penalize the crime prediction loss and treatment loss.

Observe that there is no constraint on g _α in Eq. (12), which makes the formulation ill-posed. Based on our Positivity assumption in causal inference, i.e, p(t|x) > 0, it is clear that g _α needs to satisfy the positivity assumption: g _α > 0. Besides, to prevent exploding gradients, it is important to normalize the weights across the dataset (or in the current training batch). In principle, we perform a normalization step that rescales the value of the adversary weighting component. Specifically, we make: w α = 1 + N ⋅ g α ( t | x , y ) ∑ i = 1 N g α ( t | x , y ) . (13) where N is current batch sizes. Finally, we present the objective function of the proposed minimax game in two stages: ( θ ̂ f , θ ̂ y ) = arg min J ( θ f , θ y , α ̂ t ) (14) α ̂ t = arg max J ( θ ̂ f , θ ̂ y , α t ) . (15)

Without loss of generality, we treat the objective in Eq. (12) as two component of leaner and adversary: arg min θ ̂ f , θ ̂ y J ( θ f , θ y , α ̂ t ) and arg max α ̂ t J ( θ ̂ f , θ ̂ y , α t ) . To handle the adversarial training, we adopt the optimization setup where the learner and adversary take turns to update their model. A more detailed training optimization can be found in Goodfellow et al. (2014).

We retrieve and collect the publicly available stop, search and frisk data from The New York Police Department website during 2018 January to 2019 December. This dataset serves demographic and other information about drivers stopped by NYC police force. Since police enforcement is dynamically changed. To validate the robustness of our method, we partition the each annual data into two continues subsets, e.g., ‘NYCSF_2018F(irst)’ and ‘NYCSF_2018L(ast)’, whose durations are half years. For each subset, we select the former 4-month as training set, and the following 2 month as validation and testing set respectively. We also mix the 2 year dataset as a ‘NYCSF_Mix’ dataset, then we randomly split the dataset as 7:2:1 for training, testing and validation respectively.

For data processing, since some ‘null’ value only means ‘False’ in NYCSF stops. We carefully analyze the data, and replace ‘(null)’ with ‘F’. For the remaining data, we drop the data records with default or wrong values. For the textual feature, e.g., witness reports, we initialize the lookup table for textual data with the pre-trained vectors from GloVe Pennington et al. (2014) by setting l as 300. For numerical values, we encode categorical variables with one-hot embedding.

In this section, we mainly describe baselines and evaluation metrics. For all the methods, we regard race as the sensitive attribute.

1) MLP Gardner and Dorling (1998). The vanilla model using multilayer perceptron (MLP) as the network architecture. This is the base model which does not take fairness into consideration. Since we take little attention on the model architecture in this paper and to make a fair comparison, we also use MLP as base model for other baseline.

Evaluation Metric: To measure the efficiency and fairness, following existing works in Wick et al. (2019), we adopt F-1 score as our utility metric and adopt demographic parity (DP) Feng et al. (2019) as our fairness metrics. Unlike accuracy which is easy to achieve high performance for trivial predictions, F-1 shows discrepancy of class imbalance Wick et al. (2019). We stratify the test data by its race label, compute F1 score for each racial group and report: 1) Macro-F1: macro average over all racial groups; 2) Minority F-1: the lowest F1 score reported across all racial groups. To evaluate fairness, demographic parity requires the behavior of prediction models to be fair on different sensitive groups. specifically, it requires the positive rate across sensitive attributes are equal: E ( y ̂ | S = i ) = E ( y ̂ | S = j ) , ∀ i ≠ j (16)

In the experiment, we report difference in demographic parity: Δ D P = E ( y ̂ | S = i ) = E ( y ̂ | S = j ) , ∀ i ≠ j (17)

In this section, to answer RQ2, we make a comparison with the classical fairness based methods. To make a fair comparison, we set the base classifier as a MLP for all the mentioned methods. Table 3 shows the main results, and we obtain the following observations:

• Proposed method achieves promising performance in terms of utility and fairness across all the datasets in terms of all time slot splitting. This result validates the assumption that unfairness in this task is partly due to selection biases. And our proposed method can alleviate selection for ‘stop-and-frisk’ programs.

We have provided a theoretical analysis on the proposed adversarial re-weighting learning method. In this section, we want to investigate RQ3, “are the learned weights really meaningful?”, directly through visualizing the example weights assigned by our model to four quadrants of a confusion matrix. The result is shown in Figure 3. Each subplot visualizes the learnt weight on x-axis and their corresponding density on y-axis. We obtain the following observations:

• Sensitive groups are upsampled with a lower inverse weighting score. It is clear that the density of minority groups, like Asian, occupies more in the low score field.

In this section, to answer RQ4, we analyze the sensitivity of our proposed model towards group size, to evaluate its robustness. Selection bias indicates that different races are stopped and searched differently. To replicate selection bias, we vary the fraction of the black group and under-sample the data from 0.2 to 1.0 in the training set. We report the result on the original test dataset and show the result in Figure 4. Results are reported on the NYCSF_Mix dataset, with LFR as the baseline.

It is obvious that our model and LFR is robust to selection bias, w.r.t. the number of group sizes. As the fraction of the black race sample increases, we are forced to over-sample the black people. While the fairness metric Δ _DP for LFR keeps going up. Besides, we can also observe that IPW is sensitive towards group size. This verifies our assumption of incorporating unknown factors into the formulation of propensity score estimation.