NOIRE-Net–a convolutional neural network for automatic classification and scaling of high-latitude ionograms

Abstract

Millions of ionograms are acquired annually to monitor the ionosphere. The accumulated data contain untapped information from a range of locations, multiple solar cycles, and various geomagnetic conditions. In this study, we propose the application of deep convolutional neural networks to automatically classify and scale high-latitude ionograms. A supervised approach is implemented and the networks are trained and tested using manually analyzed oblique ionograms acquired at a receiver station located in Skibotn, Norway. The classification routine categorizes the observations based on the presence or absence of E− and F-region traces, while the scaling procedure automatically defines the E− and F-region virtual distances and maximum plasma frequencies. Overall, we conclude that deep convolutional neural networks are suitable for automatic processing of ionograms, even under auroral conditions. The networks achieve an average classification accuracy of 93% ± 4% for the E-region and 86% ± 7% for the F-region. In addition, the networks obtain scientifically useful scaling parameters with median absolute deviation values of 118 kHz ±27 kHz for the E-region maximum frequency and 105 kHz ±37 kHz for the F-region maximum O-mode frequency. Predictions of the virtual distance for the E− and F-region yield median distance deviation values of 6.1 km ± 1.7 km and 8.3 km ± 2.3 km, respectively. The developed networks may facilitate EISCAT 3D and other instruments in Fennoscandia by automatic cataloging and scaling of salient ionospheric features. This data can be used to study both long-term ionospheric trends and more transient ionospheric features, such as traveling ionospheric disturbances.

1 Introduction

The ionosphere is the uppermost layer of the Earth’s atmosphere and the link between the neutral atmosphere and the plasmasphere (Russell et al., 2016). This region of the Earth’s atmosphere is partly ionized (thereof the name ionosphere) and exhibits plasma behavior. The plasma density (n_e) in the Arctic ionosphere is highly variable in both altitude and time, and balanced by different sources and sinks, such as solar radiation, particle precipitation, and recombination with neutrals (with density n_n). Figure 1 illustrates the typical ionospheric variability in altitude and time (day/night and solar minimum/maximum). The ionosphere is traditionally divided into 3 regions: the D-region (60–90 km), the E-region (90–160 km) and the F-region ( 160 km), the F-region is often subdivided into the F1 (160–180 km) and F2 ( 180 km) layers (Hunsucker and Hargreaves, 2007, p 13–15).

FIGURE 1

Nearly a century ago, Breit and Tuve (1925, 1926) provided the first experimental evidence of the ionosphere by observing that pulsed high-frequency radio waves are reflected by a conducting and variable layer in the upper atmosphere. Their observations were made by the first version of an ionosonde, a radar system that is still in active use today with more than 100 ionosondes being deployed worldwide (Xiao et al., 2020). An ionosonde consists of an antenna and a receiver that transmit and receive pulsed radio waves that are swept through the typical ionospheric plasma frequency range (0.5 MHz–20 MHz). The distance to the reflection point is calculated from the time delay of the signal from the transmitter to the receiver and is referred to as the virtual distance in this article. Radio waves transmitted vertically are reflected when the transmitted frequency (f_T) is equal to the local plasma frequency (f_p), i.e., (f_T = f_p), while waves transmitted at an angle are reflected at lower altitudes due to refraction in the ionosphere. The plasma frequency in the ionosphere is related to the local plasma density by:this expression can therefore be used to estimate the altitude profile of the plasma density (n_e(z)) up to the altitude of the maximum plasma frequency in the E− and F-region. Here, (e) is the elementary charge, (m_e) is the electron mass and (ϵ₀) is the permittivity of free space. Additionally, ionosondes transmit both left-hand and right-hand circular polarized waves, commonly referred to as the Ordinary mode (O-mode) and Extraordinary mode (X-mode) waves, respectively. Note that these mode definitions are traditionally opposite in the fields of physics and engineering (Rexer, 2021). Radio waves transmitted in these two modes are reflected at different altitudes in the ionosphere. In this work, we focus on the O-mode reflection whenever the O- and X-mode traces can be discerned in the F-region.

The ionogram is the basic data product acquired by the ionosonde. The ionogram is a 2-D visualization of the received echo as a function of transmitted frequency and virtual distance (time-of-flight). Each ionosonde typically produces an ionogram every 1–15 min, resulting in several thousand ionograms every month. This data can be used to support ionospheric observations from radars (Patra et al., 2009) and rockets (Savio Odriozola et al., 2017; Neelakshi et al., 2022) and to track both long-term ionospheric trends (Xu et al., 2004) and more transient ionospheric features, such as traveling ionospheric disturbances (Pederick et al., 2017).

Several tools have been developed to extract important features from ionograms automatically, e.g., Autoscala (Scotto and Pezzopane, 2002; Pezzopane and Scotto, 2007), Univap Digital Ionosonde Data Analysis (UDIDA) (Pillat et al., 2013) and other methods, see for example,: Ding et al. (2007); Jiang et al. (2013, 2015); Chen et al. (2018). In addition, automatic methods specialized for processing oblique ionograms have been developed, see for example,: (Ippolito et al., 2015; Song et al., 2016; Jiang et al., 2022). Automatic Real-Time Ionogram Scaling with True-heights (ARTIST), version 5, is currently the most widely used routine (Reinisch and Xueqin, 1983; Galkin and Reinisch, 2008). However, ARTIST-5 achieves sub-optimal performance compared to human scaling, in particular for low signal-to-noise ratio data (Xiao et al., 2020). Themens et al. (2022) compared the ARTIST-5 performance to ∼35 000 manually analyzed ionograms acquired by the Global Ionospheric Radio Observatory (GIRO). They concluded that ARTIST accurately extracts the F2-layer O-mode frequency and the F-region height for ionograms with high confidence scores ( 75), but is unreliable for processing ionograms with low confidence scores. The Themens et al. (2022) study further indicated that ARTIST-5 is unreliable for identifying the F1-layer, leaving out 39% of the F1-layer ionograms, and is inaccurate for scaling the F1-layer O-mode frequency, achieving performances worse than those estimated by the International Reference Ionosphere (Bilitza, Dieter et al., 2014). In addition, Stankov et al. (2023) estimated the error bounds of the scaling parameters of ARTIST-5 by comparison to more than 50 000 manually analyzed ionograms. Using a 95% confidence, they found the error bounds: E-region O-mode frequency [-0.3 MHz, +0.8 MHz], F2-layer O-mode frequency [-0.35 MHZ, +0.25 MHZ], E-region height [-6 km, +6 km] and F2-layer height [-115 km, +45 km]. Overall, these studies indicate that reliable and accurate ionogram scaling remains a challenge, especially at high latitudes.

More recently, supervised deep learning methods, in particular deep convolutional neural networks (CNNs), have been implemented for automatic ionogram analysis with promising results. Deep convolutional neural networks are versatile algorithms designed for processing grid-like data, such as image (Krizhevsky et al., 2012; Clausen and Nickisch, 2018), video (Karpathy et al., 2014; Redmon et al., 2016) or time series (Wang et al., 2017; Kvammen et al., 2023). Deep, in this context, refers to the multi-layered architecture of the convolutional neural networks, often consisting of millions of free parameters that are optimized using manually labeled observations (i.e., ionograms) to solve the task at hand. Mochalov and Mochalova (2019) used ∼40 000 manually scaled ionograms from the Parus-A ionosonde, operated by the Pushkov Institute of Terrestrial Magnetism, near Moscow, Russia, to develop a deep neural network for detecting the ionospheric trace and separating the E, F1, and F2-region traces. Xiao et al. (2020) implemented and evaluated several convolutional neural network architectures to identify the E, F1 and F2-region traces using ∼20 000 ionograms from the DPS4D ionosonde near Wuhan, China. Xiao et al. (2020) concluded that the DIAS model (based on the ResNet-50 architecture) achieved the highest performance, outperforming ARTIST and obtaining scaling accuracies close to human experts. De La Jara and Olivares (2021) used convolutional neural networks to accurately detect the ionospheric echo using ∼51 000 ionograms from the Jicamarca VIPIR ionosonde near Lima, Peru, and further developed methods to isolate the ionospheric trace from noise and interference. Rao et al. (2022) implemented a pre-trained VGG-16 convolutional neural network to classify ionograms acquired by the Canadian Advanced Digital Ionosonde (CADI) system in Hyderabad, India. Rao et al. (2022) used 40 000 ionograms to develop the VGG-16 model, achieving a classification accuracy of 97% and an F1-score of 89%, and further demonstrated that the VGG-16 network provided useful outputs even under highly disturbed geomagnetic periods. Finally, Sherstyukov et al. (2023) tested several convolutional neural network architectures for processing ionograms acquired at high latitudes. A large data set consisting of 86 500 ionograms from the Sodankylä Geophysical Observatory was used. Sherstyukov et al. (2023) concluded that the InceptionV3 architecture achieved the highest performance, achieving 87.3% E-region classification accuracy and 92.8% F-region accuracy, and frequency scaling estimates with ∼0.10 MHz mean absolute error.

Following the previous work, the main purpose of this study was to develop a deep convolutional neural network specialized for processing high-latitude oblique ionograms. The high-latitude ionosphere is different from the low- and mid-latitude ionosphere due to differences in, for example, particle precipitation, solar zenith angles and plasma convection patterns (Ratovsky et al., 2014). Thus, automatic tools developed for processing low and mid-latitude oblique ionograms are sub-optimal for processing high-latitude data. In addition, previous implementations of deep neural networks have focused mainly on automatically detecting and isolating the ionospheric trace, while we use a somewhat different approach, more similar to Sherstyukov et al. (2023). In this work, we seek to automatically classify ionograms into categories (E-region, F-region and no ionospheric trace) and further quantify the characteristic E− and F-region virtual distances and maximum frequencies. This study was further motivated by the expected increased number of high-frequency radar sounders in Fennoscandia, and a great demand for reliable monitoring of the ionosphere in relation to the anticipated EISCAT 3D incoherent scatter radar in Skibotn, Norway (McCrea et al., 2015).

2 Data acquisition and processing

This work focuses on ionograms acquired by the Network of Oblique Ionospheric Receivers Experiment (NOIRE) (Floberg, 2022). NOIRE consists of 4 oblique sounding receivers positioned in Skibotn, Oulu, Kuusamo and Ivalo, illustrated in Figure 2. These receivers are linked to the ionosonde transmitter located in Sodankylä and operated by the Sodankylä Geophysical Observatory (Kozlovsky et al., 2013; Enell et al., 2016). Each receiver station is equipped with an active magnetic loop antenna and a universal Software Radio Peripheral (USRP) platform. NOIRE acquires one ionogram on each minute, resulting in 1440 ionograms each day at all receiver stations. The ionograms have dimension (310 × 310), 310 transmitted frequencies spanning linearly between 0.5 MHz and 16 MHz and 310 range gates spanning linearly between 200 km and 1500 km.

FIGURE 2

For the purpose of this study, ionograms acquired in Skibotn between 2022-04-01 and 2023-04-01 are considered. Figure 3 summarizes the solar and geophysical background conditions during this period. The data has a good representation of the seasonal variability, with more than 1000 monthly ionograms, and a broad coverage of the solar elevation angles. However, a wider range of solar and geophysical conditions would be beneficial, e.g., over an entire solar cycle, but is unobtainable since NOIRE has only been in operation (with the current software) since 2022. The framework outlined in this article can however be used to extend the considered dataset in future work. The proposed methodology can further be used to include ionograms from other receiver stations and even other ionosondes with additional training.

FIGURE 3

2.1 Preprocessing

FIGURE 4

2.2 Manual ionogram analysis

Supervised machine learning techniques require manually labeled data to both train and test the classifiers (Goodfellow et al., 2016, p 101–102). Attention is therefore needed in order to construct a high-quality labeled data set that can be used to develop reliable classifiers without significant contamination of biases, corrupted data files and mislabeled ionograms (McKay and Kvammen, 2020). For this work, a two-step routine was employed. The initial step involves the classification of ionograms based on the features of interest, namely, the presence or absence of a clearly discernible E-region and/or F-region trace. These two binary classifications yield four categories: E-region, F-region, E and F-region and no trace. The second step is manually identifying the E-region and/or F-region virtual distance and maximum frequencies (if traces are present). Figure 5 presents examples of ionograms with E-region, F-region, E and F-region and no trace, along with the manual scaling for the positive classes. Table 1 presents the number of ionograms in each class.

FIGURE 5

In practice, each ionogram was drawn randomly from the considered data set (without replacement) and displayed. A Graphical User Interface (GUI) was then used to implement the two-step routine, making it easy for the human user to assign an ionogram class and the relevant scaling parameters rigorously and efficiently. In total, 16 776 ionograms were manually classified and scaled by 5 different scientists with knowledge of the salient ionogram features.

2.3 10-fold cross-validation

The manually analyzed data was chronologically sorted and split into training, validation and testing sets using a 10-fold cross-validation approach (Hastie et al., 2009). This method divides the sorted data into 10 subsets (or folds) with approximately equal size (Chollet, 2021; Varoquaux and Colliot, 2023). 8 folds (∼224 h of data) was used for training the convolutional neural networks, one fold (∼28 h) was used for validating the models and the last fold was held out for testing. 10 convolutional neural networks were then developed using different data subsets through circular permutation, illustrated in Figure 6, allowing the mean performance across the 10 networks to be reported with error estimates.

FIGURE 6

The 10-fold cross-validation approach was implemented to minimize the number of observations in the validation and testing sets with a high correlation (in time) to the ionograms in the training data. Highly correlated ionograms are problematic since near-consecutive observations are likely to be similar, leading to an overestimation of the reported performance (Camporeale, 2019). Note that a random split would cause approximately one-third (on average 36.2% ± 0.9%) of the validation and testing data to be highly time-correlated with the training data. Highly time-correlated observations are defined here as ionograms that are separated by no more than 10 min.

3 Convolutional neural network training

This project aimed at developing a tool that can take an ionogram as input and automatically output the ionogram class and the ionogram scaling parameters for the positive classes. Mathematically, the objective becomes to find a non-linear function (f), parameterized using the trainable weights and biases (w), that maps an input ionogram (x) to the output class or scaling parameters (y), this yields Eq. 2:

In this work, the free parameters (w) are structured in a CNN consisting of 12 convolutional layers and 3 fully connected layers, which implements the non-linear function f (w, x). For an introduction to convolutional neural network designs see e.g., Goodfellow et al. (2016, Chapter 9). The employed CNN, named NOIRE-Net, is illustrated in Figure 7 and the number of free parameters in each layer are summarized in Table 2.

FIGURE 7

The rectified linear unit (ReLU) was used as the activation function (Nair and Hinton, 2010), i.e., the function that determines the activation of the neurons in hidden layers based on the network weights and the input ionogram. Six maximum pooling layers were introduced to the network in order to downscale the data as information propagates through the network while extracting the most important features (Zhou and Chellappa, 1988). Batch normalization (Ioffe and Szegedy, 2015) was added after each convolutional layer to stabilize the learning process while dropout (Srivastava et al., 2014) was added between two of the fully connected layers to prevent overfitting. This architectural design was influenced by well-known CNN designs such as AlexNet (Krizhevsky et al., 2012) and VGG (Simonyan and Zisserman, 2015). The main difference is that NOIRE-Net was adapted to the specific input image size of the ionograms.

As commonly used for binary classification tasks with CNNs, a logistic sigmoid activation function was applied to the output of the E-classify and F-classify networks (Goodfellow et al., 2016, p 65–66), i.e., an output ≥0.5 indicate a positive class (presence of E/F) and an output 0.5 indicate a negative class (absence of E/F). For the ionogram scaling, a linear activation was used to output the E/F-region virtual distance and maximum frequency. The networks were otherwise identical. Note that a single CNN could be trained to solve all tasks, this approach requires however a multi-headed model which is more challenging to both train and test.

The parameters (w) were optimized using stochastic gradient descent (Adam algorithm) (Kingma and Ba, 2017) and the training data set, consisting of input-output pairs (x_i, y_i). For the labeling task, we used the binary cross-entropy between the distribution of true (y_i) and predicted (f (w, x_i)) labels to define the loss function (Theodoridis and Koutroumbas, 2009b, p 172–173), as expressed in Eq. 3:where denotes the optimized parameters (which are fixed after training).

For the regression task of estimating the E− or F-region virtual distance and maximum frequencies, a sum of squared errors between the manually scaled parameters (y_i) and the predictions (f (w, x_i)) was used as the loss function (Theodoridis and Koutroumbas, 2009a, p 108–109), as expressed in Eq. 4:

The CNNs were implemented using the TensorFlow and Keras frameworks. The training was conducted on a MacBook Pro with a M1 Max 32-core GPU and required about 2 h for each network. A dynamic learning rate between 0.001 and 1 ⋅10^–5 over 100 epochs with a batch size of 64 was used to optimize the network parameters. These hyper-parameters were selected after a few experimental runs with different parametrizations. The effect of varying dropout rates, optimization methods and activation functions are unexplored. A systematic tuning of all these hyper-parameters, e.g., using an automatic optimization tool like Optuna Akiba et al. (2019), might further enhance the NOIRE-Net performance, but was not attempted in the interest of time. For more information about the training and testing of NOIRE-Net, please visit the relevant Notebooks at https://github.com/AndreasKvammen/NOIRE-Net/.

4 Results and discussions

The classification and scaling performances of NOIRE-Net are evaluated using the testing data. We report the average performance from 10 independent training runs with random initialization of the NOIRE-Net weights and biases (w). For each training run, the manually labeled and scaled ionograms are split into training, validation and testing data sets with a (0.8, 0.1, 0.1) partitioning using the 10-fold cross-validation method described in Section 2.3. This approach, using the ensemble performance from 10 runs, provides a more robust performance and further allows for error estimates between the trained CNNs.

4.1 Classification performance

Table 3 presents the performance scores of NOIRE-Net for the classification task. The classifier is compared to a simple baseline probability model. The probability model randomly predicts the class of the ionogram based on the probability of a positive class (65.2% for the E-region and 44.8% for the F-region), calculated using the manually labeled data in Table 1. NOIRE-Net outperforms the probability model across all metrics. NOIRE-Net achieves a reliable E-region classification performance with very high (≥0.9) scores over all metrics, see the Appendix for the definitions of the evaluation metrics. For the F-region, NOIRE-Net archives a good performance (≥0.85) for the precision and accuracy metrics but has a lower score for the recall (0.81 ± 0.15) and F1-score (0.84 ± 0.01). This indicates that some observations that were manually labeled as F-region are misclassified (no F-region) by NOIRE-Net.

Figure 8 presents the NOIRE-Net performance as confusion matrices. Overall, NOIRE-Net achieves a reliable E-region classification performance and a good performance for the F-region classification, although with a significant (19% ± 15%) proportion of false negative F-region observations (as expected from the lower recall score).

FIGURE 8

4.2 Scaling performance

The scaling performance of NOIRE-Net is evaluated by the median absolute deviation (MAD) and the root mean square error (RMSE). Table 4 summarizes the performance for each output parameter across 10 training runs. The NOIRE-Net scaling performance is compared to a mean reference model. The mean model always predicts the mean value of the scaling parameters, 6.21 MHz for the E-region maximum frequency and 6.07 MHz for the F-region maximum O-mode frequency, and 404 km and 611 km for the virtual distances, respectively.

Overall, NOIRE-Net outperforms the mean model and provides scientifically useful outputs with MAD values of 118 kHz ±27 kHz for the E-region maximum frequency and 105 kHz ±37 kHz for the F-region maximum O-mode frequency. Predictions of the virtual distance for the E− and F-region yield MAD values of 6.1 km ± 1.7 km and 8.3 km ± 2.3 km. Note that the ionograms have dimensions (310 × 310), with 50 kHz frequency resolution and a ∼4.2 km virtual distance resolution for each pixel. Thus, NOIRE-Net provides predictions with MAD values within 3 pixels of manually scaled E and F-region traces. The RMSE values are sensitive to outliers and NOIRE-Net archives RMSE values within 10 pixels. The E-region frequency is the most uncertain parameter, indicating a significant number of outliers, possibly due to specular meteor trail echoes (Ellyett and Goldsbrough, 1976) and a large variability from quiet to auroral conditions.

4.3 Automatic ionogram classification and scaling with NOIRE-Net

In this section, the NOIRE-Net performance is studied by analyzing the stability across the parameter space and by evaluating the consistency (in time) during both quiet and active auroral conditions. These analyses are performed using additional testing data sets, auxiliary to the manually labeled and scaled data set presented in Table 1.

4.3.1 Scaling stability across the parameter space

The stability of NOIRE-Net across the parameter space is studied by analyzing the performance on an additional testing data set consisting of 652 ionograms. In this data set, each ionogram was labeled and scaled by 1-5 human experts, with a mean value of 3.92 human experts per ionogram. A summary of this data set is presented in Table 5. The NOIRE-Net scaling predictions are compared to the multi-human analysis in Figure 9. Note that ionograms in the multi-human data set (Table 5) with a high time correlation to the observations in the NOIRE-Net data set (Table 1) were excluded. In total 46.2% of the multi-human data was rejected due to high time correlation, excluding ionograms acquired within 10 min of an observation in Table 1. Still, the remaining data presented in Figure 9 contain the 48.5% semi-correlated ionograms (observations sampled within 10-20 min) and might present an overestimated performance compared to data outside the time domain of the NOIRE-Net data set.

FIGURE 9

In summary, Figure 9 indicates that NOIRE-Net provides scaling parameters that are generally in-line with the human interpretation. A few outliers can be seen, especially for the E-region maximum frequency and the F-region virtual distance, as already implied by the large RMSE values in Table 4. It should be noted that ionograms with low signal-to-noise ratios are often ambiguous, and the “correct” class and scaling parameters differ from person to person. Still, there is no clear skew between the NOIRE-Net predictions and the human consensus, suggesting that NOIRE-Net provides scientifically useful scaling results across the parameter space.

4.3.2 Classification and scaling stability in time

Consecutive ionograms acquired over a 24-h period (24 October 2022) were manually analyzed to study the self-consistency of NOIRE-Net (in time). Furthermore, this data set was used to compare NOIRE-Net and human analysis during both quiet and active auroral conditions. A summary of this data set is presented in Table 6. Ionograms from 24 October 2022 were selected for this analysis since auroral emissions were clearly visible in the Kiruna all-sky camera for a longer period (21:00-00:00) without cloud contamination or significant light pollution from the moon or other sources. Figure 10 presents the comparison of the NOIRE-Net and human classification and scaling. A movie of the NOIRE-Net performance on this data is added to the Supplementary Material.

FIGURE 10

Figure 10 demonstrates that NOIRE-Net scaling generally follows human interpretation under various conditions and further provides self-consistent outputs (in time) with few outliers. Figure 10 further suggests that inconsistency predominantly occurs for ambiguous ionograms, often with low signal-to-noise ratios, ionograms that are difficult to interpret for human experts as well. In addition, NOIRE-Net provides some disagreement with human interpretation under auroral conditions, as seen in the E-region maximum frequency scaling in Figure 10A.

Figure 11 investigates the E-region maximum frequency prediction in detail for a variable period 21:00-00:00 with auroral precipitation. The observed green-line intensity enhancement (I₅₅₇₇) is approximately proportional to the production of secondary electrons (q_e) from precipitating particles (Rees and Luckey, 1974). Further, at E-region heights during steady-state, the electron density is proportional to (Brekke, 2012; Section 7.9). Combining this with the plasma frequency expression, Eq. 1, gives the first-order relationship: , allowing the E-region maximum frequency to be compared to the intensity enhancement in the green-light camera channel, assuming that the altitude distribution of I₅₅₇₇ is roughly stable. Figure 11 shows that NOIRE-Net can detect auroral conditions as enhancements in the E-region maximum frequency, consistent with human interpretation.

FIGURE 11

5 Conclusion and future work

In this study, deep convolutional neural networks have been trained and tested for automatic classification and scaling of high-latitude oblique ionograms. Overall, the trained networks (jointly named NOIRE-Net) obtained reliable outputs in-line with human interpretation. NOIRE-Net achieved an average classification accuracy of 93% ± 4% for the E-region and 86% ± 7% for the F-region. Furthermore, NOIRE-Net obtained scientifically useful scaling parameters with median absolute deviation values of 118 kHz ±27 kHz for the E-region maximum frequency and 105 kHz ±37 kHz for the F-region maximum O-mode frequency. Predictions of the virtual distance for the E− and F-region yielded median distance deviation values of 6.1 km ± 1.7 km and 8.3 km ± 2.3 km, respectively. In general, NOIRE-Net therefore achieves scaling performances with median absolute deviation values well within the 95% confidence error bounds of ARTIST-5 (Stankov et al., 2023): E-region O-mode frequency [-0.3 MHz, +0.8 MHz], F2-layer O-mode frequency [-0.35 MHZ, +0.25 MHZ], E-region height [-6 km, +6 km] and F2-layer height [-115 km, +45 km].

The stability of NOIRE-Net over several testing data sets and under auroral conditions has been studied. Overall, NOIRE-Net provides stable classification and scaling performances. Our analysis indicates that disagreement with human interpretation primarily arises for ambiguous ionograms, often with low signal-to-noise ratios or acquired under challenging auroral conditions, i.e., under conditions that are difficult to interpret for human experts as well. An enhanced E-region scaling performance may however be achieved for periods with auroral precipitation by adding auroral conditions as an independent class in the classification network and further train a network specialized for scaling ionograms with clearly discernible auroral features. This additional routine, where auroral ionograms are explicitly cataloged, may provide a useful data product for auroral researchers using ionogram data or data from co-located instruments. Further improvements to NOIRE-Net might be achieved by enriching the training data set, e.g., by adding ionograms from other years (with different solar activity levels), and by systematically tuning the NOIRE-Net hyper-parameters.

Our study concludes that deep convolutional neural networks are suitable for classification and scaling of high-latitude ionograms. The framework outlined in this article may serve as a foundation for implementing ionograms from other NOIRE receiver stations with additional training. Automatic classification and scaling of ionograms acquired by the Network of Oblique Ionospheric Receivers Experiments (NOIRE) will be a great asset for studies of the ionosphere over Fennoscandia in general and in particular for ionospheric monitoring in relation to the anticipated EISCAT 3D incoherent scatter radar in Skibotn, Norway (McCrea et al., 2015).

References

• 1

  AkibaT.SanoS.YanaseT.OhtaT.KoyamaM. (2019). “Optuna: a next-generation hyperparameter optimization framework,” in Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (New York, NY, USA: Association for Computing Machinery), KDD ’19, Anchorage, AK, USA, August 4 - 8, 2019, 2623–2631.

  • Google Scholar

• 2

  BerginA.ChapmanS. C.GjerloevJ. W. (2020). AE, D_ST, and their SuperMAG counterparts: the effect of improved spatial resolution in geomagnetic indices. J. Geophys. Res. Space Phys.125, e2020JA027828. 10.1029/2020JA027828

  • CrossRef
  • Google Scholar

• 3

  BilitzaD.AltadillD.ZhangY.MertensC.TruhlikV.RichardsP.et al (2014). The international reference ionosphere 2012 - a model of international collaboration. J. Space Weather Space Clim.4, A07. 10.1051/swsc/2014004

  • CrossRef
  • Google Scholar

• 4

  BreitG.TuveM. A. (1925). A radio method of estimating the height of the conducting layer. Nature116, 357. 10.1038/116357a0

  • CrossRef
  • Google Scholar

• 5

  BreitG.TuveM. A. (1926). A test of the existence of the conducting layer. Phys. Rev.28, 554–575. 10.1103/PhysRev.28.554

  • CrossRef
  • Google Scholar

• 6

  BrekkeA. (2012). Physics of the upper polar atmosphere. Cham: Springer Science and Business Media.

  • Google Scholar

• 7

  CamporealeE. (2019). The challenge of machine learning in space weather: nowcasting and forecasting. Space weather.17, 1166–1207. 10.1029/2018SW002061

  • CrossRef
  • Google Scholar

• 8

  ChenZ.GongZ.ZhangF.FangG. (2018). A new ionogram automatic scaling method. Radio Sci.53, 1149–1164. 10.1029/2018RS006574

  • CrossRef
  • Google Scholar

• 9

  CholletF. (2021). Deep learning with Python (simon and schuster).

  • Google Scholar

• 10

  ClausenL. B. N.NickischH. (2018). Automatic classification of auroral images from the oslo auroral themis (oath) data set using machine learning. J. Geophys. Res. Space Phys.123, 5640–5647. 10.1029/2018JA025274

  • CrossRef
  • Google Scholar

• 11

  De La JaraC.OlivaresC. (2021). Ionospheric echo detection in digital ionograms using convolutional neural networks. Radio Sci.56, 1–15. 10.1029/2020RS007258

  • CrossRef
  • Google Scholar

• 12

  DingZ.-H.NingB.-Q.WanW.-X. (2007). Real-time automatic scaling and analysis of ionospheric ionogram parameters. Chin. J. Geophys.50, 837–847. 10.1002/cjg2.1101

  • CrossRef
  • Google Scholar

• 13

  EllyettC. D.GoldsbroughP. F. (1976). Relationship of meteors to sporadic e, 1. a sorting of facts. J. Geophys. Res. (1896-1977)81, 6131–6134. 10.1029/JA081i034p06131

  • CrossRef
  • Google Scholar

• 14

  EnellC.-F.KozlovskyA.TurunenT.UlichT.VälitaloS.ScottoC.et al (2016). Comparison between manual scaling and autoscala automatic scaling applied to sodankylä geophysical observatory ionograms. Geoscientific Instrum. Methods Data Syst.5, 53–64. 10.5194/gi-5-53-2016

  • CrossRef
  • Google Scholar

• 15

  FlobergJ. (2022). “Design and implementation of an oblique ionosonde receiver,” in For studies of spatial and temporal ionospheric structures (Tromsø, Norway: UiT Norges arktiske universitet). Master’s thesis.

  • Google Scholar

• 16

  GalkinI. A.ReinischB. W. (2008). The new artist 5 for all digisondes. Ionosonde Netw. Advis. Group Bull.69, 1–8.

  • Google Scholar

• 17

  GavrikovP. (2020). visualkeras. Available at: https://github.com/paulgavrikov/visualkeras.

  • Google Scholar

• 18

  GoodfellowI.BengioY.CourvilleA. (2016). Deep learning. Massachusetts, United States: MIT Press. Available at: http://www.deeplearningbook.org.

  • Google Scholar

• 19

  HastieT.TibshiraniR.FriedmanJ. H.FriedmanJ. H. (2009). The elements of statistical learning: data mining, inference, and prediction, vol. 2. Cham: Springer. 10.1007/978-0-387-21606-5

  • CrossRef
  • Google Scholar

• 20

  HunsuckerR. D.HargreavesJ. K. (2007). The high-latitude ionosphere and its effects on radio propagation. Cambridge University Press.

  • Google Scholar

• 21

  IoffeS.SzegedyC. (2015). “Batch normalization: accelerating deep network training by reducing internal covariate shift,” in Proceedings of the 32nd International Conference on Machine Learning, (Lille, France: PMLR), vol. 37 of Proceedings of Machine Learning Research, Lille, France, 6-11 July 2015, 448–456.

  • Google Scholar

• 22

  IppolitoA.ScottoC.FrancisM.SettimiA.CesaroniC. (2015). Automatic interpretation of oblique ionograms. Adv. Space Res.55, 1624–1629. 10.1016/j.asr.2014.12.025

  • CrossRef
  • Google Scholar

• 23

  JiangC.YangG.LanT.ZhuP.SongH.ZhouC.et al (2015). Improvement of automatic scaling of vertical incidence ionograms by simulated annealing. J. Atmos. Solar-Terrestrial Phys.133, 178–184. 10.1016/j.jastp.2015.09.002

  • CrossRef
  • Google Scholar

• 24

  JiangC.YangG.ZhaoZ.ZhangY.ZhuP.SunH. (2013). An automatic scaling technique for obtaining f2 parameters and f1 critical frequency from vertical incidence ionograms. Radio Sci.48, 739–751. 10.1002/2013RS005223

  • CrossRef
  • Google Scholar

• 25

  JiangC.ZhaoC.ZhangX.LiuT.ChenZ.YangG.et al (2022). A method for automatic inversion of oblique ionograms. Remote Sens.14, 1671. 10.3390/rs14071671

  • CrossRef
  • Google Scholar

• 26

  KarpathyA.TodericiG.ShettyS.LeungT.SukthankarR.Fei-FeiL. (2014). “Large-scale video classification with convolutional neural networks,” in 2014 IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA, June 23 2014 to June 28 2014, 1725–1732.

  • Google Scholar

• 27

  KingJ. H.PapitashviliN. E. (2005). Solar wind spatial scales in and comparisons of hourly wind and ace plasma and magnetic field data. J. Geophys. Res. Space Phys.110. 10.1029/2004JA010649

  • CrossRef
  • Google Scholar

• 28

  KingmaD. P.BaJ. (2017). Adam: a method for stochastic optimization. Available at: https://arxiv.org/abs/1412.6980.

  • Google Scholar

• 29

  KozlovskyA.TurunenT.UlichT. (2013). Rapid-run ionosonde observations of traveling ionospheric disturbances in the auroral ionosphere. J. Geophys. Res. Space Phys.118, 5265–5276. 10.1002/jgra.50474

  • CrossRef
  • Google Scholar

• 30

  KrizhevskyA.SutskeverI.HintonG. E. (2012). Imagenet classification with deep convolutional neural networks. Adv. neural Inf. Process. Syst.25.

  • Google Scholar

• 31

  KvammenA.WickstrømK.KociscakS.VaverkaJ.NouzakL.ZaslavskyA.et al (2023). Machine learning detection of dust impact signals observed by the solar orbiter. Ann. Geophys.41, 69–86. 10.5194/angeo-41-69-2023

  • CrossRef
  • Google Scholar

• 32

  McCreaI.AikioA.AlfonsiL.BelovaE.BuchertS.ClilverdM.et al (2015). The science case for the eiscat_3d radar. Prog. Earth Planet. Sci.2, 21. 10.1186/s40645-015-0051-8

  • CrossRef
  • Google Scholar

• 33

  McKayD.KvammenA. (2020). Auroral classification ergonomics and the implications for machine learning. Geoscientific Instrum. Methods Data Syst.9, 267–273. 10.5194/gi-9-267-2020

  • CrossRef
  • Google Scholar

• 34

  MochalovV.MochalovaA. (2019). Application of deep learning to recognize ionograms. 2019 Russ. Open Conf. Radio Wave Propag. (RWP)1, 477–479. 10.1109/RWP.2019.8810326

  • CrossRef
  • Google Scholar

• 35

  NairV.HintonG. E. (2010). “Rectified linear units improve restricted Boltzmann machines,” in Proceedings of the 27th International Conference on International Conference on Machine Learning (Madison, WI, USA: Omnipress), ICML’10, Madison, Wisconsin, USA, July 24-27, 1998, 807–814.

  • Google Scholar

• 36

  NeelakshiJ.RosaR. R.SavioS.StephanyS.de MenesesF. C.KheraniE. A.et al (2022). Multifractal characteristics of the low latitude equatorial ionospheric e–f valley region irregularities. Chaos, Solit. Fractals156, 111808. 10.1016/j.chaos.2022.111808

  • CrossRef
  • Google Scholar

• 37

  NewellP. T.GjerloevJ. W. (2011). Evaluation of supermag auroral electrojet indices as indicators of substorms and auroral power. J. Geophys. Res. Space Phys.116. 10.1029/2011JA016779

  • CrossRef
  • Google Scholar

• 38

  PatraA. K.Venkateswara RaoN.PhanikumarD. V.ChandraH.DasU.SinhaH. S. S.et al (2009). A study on the low-latitude daytime e region plasma irregularities using coordinated vhf radar, rocket-borne, and ionosonde observations. J. Geophys. Res. Space Phys.114. 10.1029/2009JA014501

  • CrossRef
  • Google Scholar

• 39

  PederickL. H.CerveraM. A.HarrisT. J. (2017). Interpreting observations of large-scale traveling ionospheric disturbances by ionospheric sounders. J. Geophys. Res. Space Phys.122, 12,556–12,569. 10.1002/2017JA024337

  • CrossRef
  • Google Scholar

• 40

  PezzopaneM.ScottoC. (2007). Automatic scaling of critical frequency fof2 and muf(3000)f2: a comparison between autoscala and artist 4.5 on rome data. Radio Sci.42, 1–17. 10.1029/2006RS003581

  • CrossRef
  • Google Scholar

• 41

  PillatV. G.GuimarãesL. N. F.FagundesP. R.da SilvaJ. D. S. (2013). A computational tool for ionosonde cadi’s ionogram analysis. Comput. Geosciences52, 372–378. 10.1016/j.cageo.2012.11.009

  • CrossRef
  • Google Scholar

• 42

  RaoT. V.SridharM.RatnamD. V. (2022). An automatic cadi’s ionogram scaling software tool for large ionograms data analytics. IEEE Access10, 22161–22168. 10.1109/ACCESS.2022.3153470

  • CrossRef
  • Google Scholar

• 43

  RatovskyK.ShiJ.OinatsA.RomanovaE. (2014). Comparative study of high-latitude, mid-latitude and low-latitude ionosphere on basis of local empirical models. Adv. Space Res.54, 509–516. Recent Advances in Equatorial, Low- and Mid-Latitude Mesosphere, Thermosphere-Ionosphere System Studies. 10.1016/j.asr.2014.02.019

  • CrossRef
  • Google Scholar

• 44

  RedmonJ.DivvalaS.GirshickR.FarhadiA. (2016). You only look once: unified, real-time object detection.

  • Google Scholar

• 45

  ReesM. H.LuckeyD. (1974). Auroral electron energy derived from ratio of spectroscopic emissions 1. model computations. J. Geophys. Res. (1896-1977)79, 5181–5186. 10.1029/JA079i034p05181

  • CrossRef
  • Google Scholar

• 46

  ReinischB. W.XueqinH. (1983). Automatic calculation of electron density profiles from digital ionograms: 3. processing of bottomside ionograms. Radio Sci.18, 477–492. 10.1029/RS018i003p00477

  • CrossRef
  • Google Scholar

• 47

  RexerT. (2021). Radio wave propagation through the ionosphere. UiT Norges arktiske universitet. Ph.D. thesis.

  • Google Scholar

• 48

  RussellC. T.LuhmannJ. G.StrangewayR. J. (2016). Space physics: an introduction. Cambridge University Press.

  • Google Scholar

• 49

  Savio OdriozolaS.de MenesesF. C.MuralikrishnaP.PimentaA. A.KheraniE. A. (2017). Rocket in situ observation of equatorial plasma irregularities in the region between e and f layers over brazil. Ann. Geophys.35, 413–422. 10.5194/angeo-35-413-2017

  • CrossRef
  • Google Scholar

• 50

  ScottoC.PezzopaneM. (2002). A software for automatic scaling of fof2 and muf(3000)f2 from ionograms.

  • Google Scholar

• 51

  SherstyukovR.KozlovskyA.UlichT.MogesS. (2023). “Deep learning for ionogram parameters scaling at polar region ionosphere,” in 2023 XXXVth General Assembly and Scientific Symposium of the International Union of Radio Science (URSI GASS), Sapporo, Hokkaido, Japan, 19 - 26 August 2023, 1–4.

  • Google Scholar

• 52

  SimonyanK.ZissermanA. (2015) Very deep convolutional networks for large-scale image recognition.

  • Google Scholar

• 53

  SongH.HuY.JiangC.ZhouC.ZhaoZ.ZouX. (2016). An automatic scaling method for obtaining the trace and parameters from oblique ionogram based on hybrid genetic algorithm. Radio Sci.51, 1838–1854. 10.1002/2016RS005987

  • CrossRef
  • Google Scholar

• 54

  SrivastavaN.HintonG.KrizhevskyA.SutskeverI.SalakhutdinovR. (2014). Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res.15, 1929–1958.

  • Google Scholar

• 55

  StankovS. M.VerhulstT. G. W.SapundjievD. (2023). Automatic ionospheric weather monitoring with dps-4d ionosonde and artist-5 autoscaler: system performance at a mid-latitude observatory. Radio Sci.58, 1–20. 10.1029/2022RS007628

  • CrossRef
  • Google Scholar

• 56

  ThemensD. R.ReidB.ElvidgeS. (2022). Artist ionogram autoscaling confidence scores: best practices. URSI Radio Sci. Lett.4, 22–0001. 10.46620/22-0001

  • CrossRef
  • Google Scholar

• 57

  TheodoridisS.KoutroumbasK. (2009a). “Chapter 3 - linear classifiers,” in Pattern recognition Editors TheodoridisS.KoutroumbasK.4 (Boston: Academic Press), 91–150. 10.1016/B978-1-59749-272-0.50004-9

  • CrossRef
  • Google Scholar

• 58

  TheodoridisS.KoutroumbasK. (2009b). “Chapter 4 - nonlinear classifiers,” in Pattern recognition Editors TheodoridisS.KoutroumbasK.4 (Boston: Academic Press), 151–260. 10.1016/B978-1-59749-272-0.50004-9

  • CrossRef
  • Google Scholar

• 59

  VaroquauxG.ColliotO. (2023). Evaluating machine learning models and their diagnostic value. Mach. Learn. Brain Disord., 601–630. 10.1007/978-1-0716-3195-9_20

  • CrossRef
  • Google Scholar

• 60

  WangZ.YanW.OatesT. (2017). “Time series classification from scratch with deep neural networks: a strong baseline,” in 2017 International Joint Conference on Neural Networks (IJCNN), Anchorage, AK, USA, May 14-19, 2017, 1578–1585.

  • Google Scholar

• 61

  XiaoZ.WangJ.LiJ.ZhaoB.HuL.LiuL. (2020). Deep-learning for ionogram automatic scaling. Adv. Space Res.66, 942–950. 10.1016/j.asr.2020.05.009

  • CrossRef
  • Google Scholar

• 62

  XuZ.-W.WuJ.IgarashiK.KatoH.WuZ.-S. (2004). Long-term ionospheric trends based on ground-based ionosonde observations at kokubunji, Japan. J. Geophys. Res. Space Phys.109. 10.1029/2004JA010572

  • CrossRef
  • Google Scholar

• 63

  Zhou and Chellappa (1988). “Computation of optical flow using a neural network,” in IEEE 1988 International Conference on Neural Networks vol.2, San Diego, CA, USA, 24-27 July 1988, 71–78.

  • Google Scholar