A Noise Filtering Algorithm for Event-Based Asynchronous Change Detection Image Sensors on TrueNorth and Its Implementation on TrueNorth

Asynchronous event-based sensors, or “silicon retinae,” are a new class of vision sensors inspired by biological vision systems. The output of these sensors often contains a significant number of noise events along with the signal. Filtering these noise events is a common preprocessing step before using the data for tasks such as tracking and classification. This paper presents a novel spiking neural network-based approach to filtering noise events from data captured by an Asynchronous Time-based Image Sensor on a neuromorphic processor, the IBM TrueNorth Neurosynaptic System. The significant contribution of this work is that it demonstrates our proposed filtering algorithm outperforms the traditional nearest neighbor noise filter in achieving higher signal to noise ratio (~10 dB higher) and retaining the events related to signal (~3X more). In addition, for our envisioned application of object tracking and classification under some parameter settings, it can also generate some of the missing events in the spatial neighborhood of the signal for all classes of moving objects in the data which are unattainable using the nearest neighbor filter.

: Mapping of ATIS pixels to TN cores for filter size 3 × 3. (A) The entire pixel plane is divided in to 12 × 12 patches, which are mapped to a single core on TrueNorth. (B) The boundary pixels of the 12 × 12 patch need to be connected with neighboring TrueNorth cores as well and, hence, are logically treated separately. (C) The 256 input axons of a core are shown as a 16 × 16 two-dimensional patch. The yellow axons directly connect with pixels from same patch while pink axons connect to neurons in the same core. These correspond to those boundary pixels of the patch that have to be split into multiple copies by neurons in this core. Blue axons connect to neurons from neighboring cores while green axons connect to input pixels, which need to be split. (D) The 256 output neurons are also shown as a 16 × 16 patch. The 12 × 12 biscuit color neurons in the centre produce the 144 filtered outputs of the 12 × 12 patch mapped to the core while the green neurons at the periphery produce the split outputs to be passed to neighboring cores. The same color coding is used in the picture of input axon mapping to make it clear which output neurons from neighboring cores connect with which input axons.
We refer to the number of neighboring patches connected to this pixel as a share-factor. Hence, internal, off-corner and corner pixels have share-factor equal to 0, 1 and 3, respectively. This also determines the number of neurons needed for each of the pixels as equal to the share-factor + 1 to copy this input as many times as needed. Now, we can quantify the total number of inputs that can be mapped to the 256 available input axons on a TrueNorth core. Denoting the number of axons for internal, off-corner and corner pixels by A int , A of f −corner and A corner , respectively, we get: A int is shown in yellow in Figure 1C. The contribution of A of f −corner is split into two colours: the inputs from pixels going to splitter neurons are shown in dark green while the output of the splitter neurons mapped back to this core are shown in magenta. Similarly, for A corner , the inputs from pixels going to splitter neurons is shown in light green while the output of the splitter neurons mapped back to this core are shown in magenta. In addition, just like this core provides inputs to neighbouring cores through splitter neurons, it must also accept inputs from its neighbours. Using A neigh to denote the number of axons connected to splitter neuron outputs from neighbouring cores, we get: A neigh is denoted in blue in Figure 1C. Finally, putting everything together, we get an inequality constraining patch size M as: We can write similar equations for the number of output neurons. Using N int , N of f −corner and N corner to denote the neurons producing outputs for the patch, splitting off-corner inputs and splitting corner inputs respectively, we get: These output neurons are shown in Figure 1D Simplifying inequalities (6) and (7), we get the same inequality From the above inequality (8), for L = 3, 5, 7, maximum M values are 12, 8, 4 respectively.

Scaling of Image size and Filter size on True North
where x denotes the least integer larger than x and 2 4(L − 1) 2 ) + 256 − 2(L − 1) is the maximum patch width for a particular filter width L obtained by solving the inequality in (19). The current mapping evaluates 2 4(L − 1) 2 ) + 256 − 2(L − 1) × 2 4(L − 1) 2 ) + 256 − 2(L − 1) neuronal filtering operations per core and for a 3 × 3 filter, it computes 144 filter operations per core. Figure plots the number of cores for these three filter sizes as a function of image sensor width P, where we assume a square sensor with resolution P × P . As a numeric example, for the 304 × 240 resolution of the ATIS with a 3 × 3 filter, 520 cores were utilized, which is approximately one-eighth of the entire TrueNorth chip resources.

Computational Requirements for Noise Filtering
For the NNb filter, the number of operations involved per event depends on the size of the filter. We need to store an array T last of the same size as the image sensor to keep track of last time an event happened at a particular location. For a L × L filter, whenever a new event occurs at pixel location (x, y) at time t, T last (x, y) has to be updated to t and this time stamp needs to be compared with the L 2 time stamps from the T last array in its neighbourhood. Hence, the number of operations required to process this event is L 2 + 1.
For the NeuNN filter, we consider an optimised FPGA or microprocessor implementation that can be event-driven and not operated on a clock tick like TrueNorth. Here, we need to store two arrays of the size of the image sensor resolution. The first one stores time of last spike T last for the neurons (and not the spike time of input pixel like the earlier NNb filter case) while the second array stores membrane potential V mem of the neuron. For an L × L filter, whenever a new event occurs at pixel location (x, y) at time t, we need to update the membrane potential due this event at the neighbourhood pixels surrounding this pixel (x, y). We also need to compare each updated V mem with the threshold to determine if a spike is generated. If so, then corresponding