MMS SITL Ground Loop: Automating the Burst Data Selection Process

Global-scale energy flow throughout Earth’s magnetosphere is catalyzed by processes that occur at Earth’s magnetopause (MP). Magnetic reconnection is one process responsible for solar wind entry into and global convection within the magnetosphere, and the MP location, orientation, and motion have an impact on the dynamics. Statistical studies that focus on these and other MP phenomena and characteristics inherently require MP identification in their event search criteria, a task that can be automated using machine learning so that more man hours can be spent on research and analysis. We introduce a Long-Short Term Memory (LSTM) Recurrent Neural Network model to detect MP crossings and assist studies of energy transfer into the magnetosphere. As its first application, the LSTM has been implemented into the operational data stream of the Magnetospheric Multiscale (MMS) mission. MMS focuses on the electron diffusion region of reconnection, where electron dynamics break magnetic field lines and plasma is energized. MMS employs automated burst triggers onboard the spacecraft and a Scientist-in-the-Loop (SITL) on the ground to select intervals likely to contain diffusion regions. Only low-resolution survey data is available to the SITL, which is insufficient to resolve electron dynamics. A strategy for the SITL, then, is to select all MP crossings. Of all 219 SITL selections classified as MP crossings during the first five months of model operations, the model predicted 166 (76%) of them, and of all 360 model predictions, 257 (71%) were selected by the SITL. Most predictions that were not classified as MP crossings by the SITL were still MP-like, in that the intervals contained mixed magnetosheath and magnetospheric plasmas. The LSTM model and its predictions are public to ease the burden of arduous event searches involving the MP, including those for EDRs. For MMS, this helps free up mission operation costs by consolidating manual classification processes into automated routines.


Statistical Study: SROI 1
histograms the percent overlap between individual GLS (left) or ABS (center) burst selections with selections made by the SITL, while overlap between the GLS and ABS is on the right. Panels (a-f) depict the overlap using all selections. Panels (h-m) filter the SITL segments to contain only those that were classified as MP encounters. ABS segments were filtered so that only those that overlapped with SITL MP crossings were kept. Similar plots for SROI3 and both SROI1 and SROI3 together are included in the supplemental material (Figs. S2 and S3).

GLS-SITL Comparison
Focusing on the GLS and SITL first, we see that if the GLS and SITL make the same selection, the SITL generally selects 100% of the GLS segment (Fig. S1a). Overall, the SITL selects 71% of all 360 GLS selections. When we take the inverse case, however, the GLS selects only 28% of SITL selections (Fig. S1d), a clear indication that the SITL is selecting a lot more than just MP crossings.
Since the GLS is trained to select only MP crossings, we next examine how many of the GLS selections overlap with intervals identified by the SITL as MP crossings. Figure S1g shows that 48% of the GLS segments were classified as MP crossings by the SITL. This can be explained partly because the SITL is aware external control factors such as telemetry restrictions and partly because some GLS segments are MP-like but are not classified as MP crossings by the SITL.
Another indicator of good model performance is that the GLS selected 76% of the 219 segments classified as MP crossings by the SITL (Fig S1j). It serves as an indicator that individual SITL classification tasks (e.g. Table 3) can be automated by ML models and that a combination of models could reduce or eliminate operations costs associated with the SITL.

ABS-SITL Comparison
Next, we compare the ABS to all SITL selections (Fig. S1b,e), and to only those SITL selections that were classified as MP crossings (( Fig. S1h,k). The SITL selects a larger percentage (82% vs. 72%) of ABS segments than GLS segments (Fig. S1b), but the ABS selects as few SITL segments as the GLS (Fig. S1e), again indicating that the SITL is making more selections than the ABS. Examining the MP crossings, only 19% of ABS segments were classified as MP by the SITL (Fig S1h). Conversely, only 32% of SITL-classified MP crossings were selected by the ABS (Fig S1k). So, while a majority of ABS selections are of interest to the SITL, the ABS is significantly under-selecting both in a general sense and with respect to MP crossings.

GLS-ABS Comparison
Lastly, we compare GLS selections with those of the ABS, and with the subset of ABS selections that overlapped with SITL MP selections. Figures S1c,f show that there is little overlap between the GLS and ABS. This is partly because of the 278 burst segments selected by the ABS, only 53 overlapped with SITL-classified MP crossings (Fig. S1l). When both the SITL and ABS select a MP crossing, the GLS does so also 79% of the time. These results indicate that, although they under-select compared to all SITL selections, the GLS and ABS are not redundant; they each make useful, complementary selections. Figure S1. Comparison between of the GLS, SITL, and ABS selections showing that the GLS outperforms the ABS at selecting SITL-classified MP crossings. Columns 1-3 show overlap between the GLS and SITL, ABS and SITL, and GLS and ABS, respectively, for all selections (a-f) and only those classified as MP crossings by the SITL (g-l). Figure S2 shows segment overlap between the GLS, ABS, and SITL for all burst selections made in SROI3, formatted the same as Figure S1. The SITL selects a large portion of both ABS and GLS selections (Fig. S2a,b), but the ABS and GLS under-select when compared to all selections made by the SITL (Fig. S2d,e).

Statistical Study: SROI 3
Extracting burst segments labeled as MP crossings by the SITL, few of either the GLS or ABS segments are classified by the SITL as MP crossings (Fig. S2g,h), similar to Figure S1g,h; however, unlike the SROI1 Figure S2. Overlap between burst segments made in SROI3, formatted the same as Figure S1.
selections, few SITL-classified MP crossings were selected by the GLS in SROI3 ( Figure S1j). As stated in the text, the GLS is making selections of MP-like intervals but in SROI3 fails to select the MP.
As for the GLS and ABS comparison, again there is very little overlap between the two sets of selections (Fig. S2c,f,i,l), indicating that they complement one another (Fig. S2a,b). Figure S3 show the overlap between GLS, ABS, and SITL selections for both SROI1 and SROI3, and is formatted the same as Figure S1. Generally, the trends are the same as for Figure S1 except that the smaller number of selections made in SROI3 weight the segments with no overlap slightly heavier. Figure S3. Overlap between burst segments made in both SROI1 and SROI3, formatted the same as Figure S1.