ORIGINAL RESEARCH article

Front. Earth Sci., 16 December 2024

Sec. Solid Earth Geophysics

Volume 12 - 2024 | https://doi.org/10.3389/feart.2024.1510962

Inverting magnetotelluric data using a physics-guided auto-encoder with scaling laws extension

  • School of Earth Sciences, Zhejiang University, Hangzhou, China

Abstract

Artificial neural networks (ANN) have gained significant attention in magnetotelluric (MT) inversions due to their ability to generate rapid inversion results compared to traditional methods. While a well-trained ANN can deliver near-instantaneous results, offering substantial computational advantages, its practical application is often limited by difficulties in accurately fitting observed data. To address this limitation, we introduce a novel approach that customizes an auto-encoder (AE) whose decoder is replaced with the MT forward operator. This integration accounts for the governing physical laws of MT and compels the ANN to focus not only on learning the statistical relationships from data but also on producing physically consistent results. Moreover, because ANN-based inversions are sensitive to variations in observation systems, we employ scaling laws to transform real-world observation systems into formats compatible with the trained ANN. Synthetic and real-world examples show that our scheme can recover comparable results with higher computational efficiency compared to the classic Occam’s inversion. This study not only perfectly fits the observed data but also enhances the adaptability and efficiency of ANN-based inversions in complex real-world environments.

1 Introduction

Magnetotelluric (MT) inverse problems are inherently nonlinear and fail to produce a unique solution due to the limitations of observed data, the presence of noise, and the model’s inherent null space (; ). Researchers typically employ deterministic methods (; ; ; ; ; ; ) or stochastic approaches (; ; ; ) to solve the inverse problems.

With the explosive development of deep learning, artificial neural network (ANN) (; ) has shown considerable promise in MT inversion (; ; ; ; ; ). ANN is a computational model inspired by the structure and function of the human brain. It consists of interconnected neurons that work together to process complex information (; ; ). Such inversions have primarily relied on classic supervised learning (; ; ), which is fundamentally underpinned by a mechanism akin to prior sampling-based strategies (). Generally, ANN learns a mapping from the feature space to the label space based on provided feature-label pairs. Then, we can apply this mapping to new features. Therefore, we need to construct a large number of geoelectric models serving as labels and estimate corresponding MT responses serving as features before training an ANN. The estimation of MT responses, typically generated through forward simulations, is straightforward. However, the construction of geoelectric models is challenging due to the complexity of Earth’s structures.

Considering the difficulty of model construction, there are two primary research directions. The first is integrating prior information from sources, such as logging, near-surface geology, and geostatistical data (; ; ). Alternatively, we can build geoelectric models using simple mathematical constraints, such as the assumption that the geoelectric model should be smooth (; ), which is consistent with traditional linearized inversion results. The former is beneficial in improving the inversion resolution, and the latter is more generic for different structures of the Earth. Moreover, developed a neural network based on deformable convolution (), which extracts hidden relationships and allows flexible adjustments in the size and shape of the feature region. utilized a self-attention mechanism () to enhance feature extraction. The supervised ANNs promote the development of MT inversions but have limitations in physical constraints and generalization: the MT response of the model recovered from the trained ANN may not fit the observed data, and we cannot reuse the trained ANN to invert new MT data, which are collected from a different observation system.

The auto-encoder (AE) can be considered a variant of supervised learning. It comprises two key components: an encoder that transforms input features into a user-specified parameter space and a decoder that reconstructs the input features from the specified space. The AE can be trained without labels by minimizing the difference between the reconstructed predictions and the original input features. To fit MT data better, researchers replaced the decoder with the MT forward operator to create a customized AE (; ; ). In this configuration, the observed MT data serves as the input to the encoder, while the encoder’s output becomes the input to the MT forward operator, which generates the predicted data. Thus, the encoder’s output represents the geoelectric model. Since the MT forward operator is governed by established physical laws, the system is referred to as a physics-guided AE (PGAE). The framework is utilized not only in the MT method but also in other geophysical methods, such as seismic (; ), geosteering (; ), and transient electromagnetic methods (; ).

MT data typically cover a frequency range spanning more than eight orders of magnitude ( Hz Hz). For real-world applications, varying frequency ranges may be used, which complicates the application of the trained ANN. Therefore, it is crucial to enhance the reusability of well-trained ANNs so that they can effectively handle diverse real-world data scenarios. In AVO (amplitude versus offset) inversion of seismic data, applied a transfer learning strategy () to extend ANN reusability across different systems, while in MT forward simulation, used transfer learning to improve training efficiency. However, such applications have yet to be reported for MT inversion.

In this study, we employ PGAE for the 1D inversion of MT data, which enables the inverted model to give better data fitting. At the same time, we incorporate scaling laws (; ; ) to enhance the reusability of the trained ANN. Firstly, we introduce the fundamental theory for the MT method, PGAE, and scaling laws in Section 2. Subsequently, we demonstrate the efficacy of the proposed approach through applications to both synthetic and real-world MT data in Section 3. Finally, we provide a critical evaluation of the strengths and limitations of the proposed method.

2 Methodology

2.1 MT method

The MT method is a passive-source geophysical technique used to investigate the resistivity distribution of the Earth’s subsurface by analyzing the natural electromagnetic fields at the surface (; ). These fields mainly originate from regional and global thunderstorm activity, as well as the interaction of solar wind with the Earth’s magnetosphere. As electromagnetic waves propagate through the subsurface, their penetration depth is affected by the diffusion of waves, which varies with frequency. Specifically, waves with lower frequencies demonstrate a greater capacity for deeper penetration into geological structures. Since MT sources are sufficiently distant and the Earth possesses a higher refractive index than air, electromagnetic waves are assumed to propagate as planar waves perpendicular to the Earth’s surface. However, the diffusion process through various subsurface structures can exhibit arbitrary polarization. The MT method typically measures the orthogonal components of the natural electric and magnetic fields at the Earth’s surface, and a complex impedance tensor is used to describe the mathematical relationship (Equation 1) between the electric and magnetic fields ():where and are the electric and magnetic fields, respectively.

The apparent resistivity and phase are defined as Equations 2, 3, respectively.where and are real and imaginary components of , is the magnetic permeability of free space, and . is the frequency.

2.2 Physics-guided auto-encoder

Building on the AE framework, we replace the decoder with the MT forward operator to create a customized ANN (Figure 1). The loss function (Equation 5) in PGAE, representing the difference between input features and output predictions , corresponds to the data misfit typically employed in traditional inversions. Moreover, a regularization term (Equation 6) is employed to ensure the smoothness of the predicted geoelectric model .where is the batch size used in training, is the forward operator, is the regularization parameter, and is a set of network parameters.

FIGURE 1

Algorithm 1 outlines the workflow for implementing PGAE. The process begins by initializing the encoder parameters using a truncated normal distribution, which helps mitigate the exploding or vanishing gradient problems during training . After initialization, the training samples are shuffled randomly, and a batch of samples is selected and fed into the encoder to generate the corresponding geoelectric models. The MT forward operator is then applied to produce the predicted MT data. Subsequently, the loss function is evaluated using Equation 4. These steps constitute the forward propagation phase. Next, the gradient of the loss function is computed with respect to the network parameters , and the parameters are updated using a suitable optimization algorithm, such as the Adam optimizer introduced by . These steps constitute the back propagation phase. If the value of the loss function, , falls below a predefined threshold, , the network is saved. Otherwise, the training loop continues until convergence.

Algorithm 1

  • Input: Initialize ANN weights , target loss , epochs, ;

  • 1whiledo

  • 2  Shuffle the samples randomly and divide into batches;

  • 3foreachbatchdo

  • 4   Compute and using Equation 7;

  • 5   Compute using Equation 4;

  • 6   Compute the gradient of the loss ;

  • 7   Update ;

  • 8ifthen

  • 9    save and break;

The PGAE training algorithm.

2.3 Scaling laws

Scaling laws provide a crucial theoretical framework for simulating the electromagnetic response of real-world resistivity structures within a laboratory setting (; ; ). Building on the foundational work of , we utilize the coordinates to represent any point in the real-world system and the coordinates to denote the corresponding point in the model-scale system. The two systems are related through a length scale factor, denoted as , and the transformation between them is governed by Equation 8.

Ignoring the displacement current, the MT field is described by the diffusive forms of Maxwell’s equations with a time-dependent component . For the real-world system, we havewhere is the electrical conductivity.

For the model-scale system, we have

Generally, we want the fields from the real-world system to be linearly transformed from the model-scale system, that iswhere and are scale factors for the electric and magnetic fields, respectively. Therefore,

Putting Equations 916 together, we have

According to Equations 17, 18, to simulate the MT response of real-world resistivity structures in a model-scale system, we should ensure that the model-scale system has the same induction number as the real-world system:where is any characteristic dimension in the real-world system and is the corresponding dimension in the model-scale system.

Scaling laws enable us to model large-scale MT responses using a small-size model. While researchers generally employ these principles for physical modeling, they are also applicable to numerical modeling. When we have an ANN trained for audio-frequency MT tasks, we can leverage the trained network to invert long-period MT data by applying scaling laws, rather than training a new network from scratch. Additionally, discrepancies between the number of frequencies in the real-world data and those used in training present another challenge. A practical solution is to interpolate the frequency sampling of the real-world data to match that of the training set. The interpolation method will perform effectively since MT apparent resistivity curves are typically smooth.

3 Numerical experiments

Recent research has established the effectiveness of employing ANNs to address the 1D MT inverse problem with a fixed observation system. Nonetheless, the application of ANNs to 2D and 3D problems remains largely exploratory. Therefore, we demonstrate the integration of PGAE and scaling laws in the 1D inversion of both synthetic and real-world MT data, highlighting their potential for enhancing inversion accuracy and generalization capabilities.

3.1 Obtaining a well-trained ANN

To prepare the training dataset, we analytically compute the apparent resistivity curves for 1D layered Earth models consisting of 31 layers and 25 frequencies, ranging from 10,000 Hz–1 Hz. The thicknesses of the layers begin at 20 m and increase logarithmically, as described in Equation 20, with the final layer always being a homogeneous half-space. For the resistivity of each layer, we uniformly sample values within the range of 0.1–100,000 in the logarithmic domain and subsequently smooth the geoelectric model using a Gaussian filter (). A total of 120,000 samples are generated, with 100,000 allocated for training and 20,000 for evaluation. Some random smooth geoelectric models can be found in Figure 2.where represents the number of layers and denotes the thickness of the layer.

FIGURE 2

.

The primary objective of this study is to enhance the generalization capabilities of the trained ANN rather than to design a high-performance ANN. Accordingly, we employ an ANN architecture that is as simple as possible. Specifically, we construct an ANN with a single hidden layer containing 500 neurons. The number of neurons in the input (output) layer corresponds to the MT data (geoelectric model) used in the training dataset. The regularization parameter in Equation 4 is set to 4.5, determined through trial and error. We code the whole PGAE framework using TensorFlow (an open-source Python library by Google) so that automatic differentiation () can be employed to calculate the gradient of the loss function.

We use the Adam optimizer to train the established network, as illustrated in Figure 3A, the data misfit decreases rapidly as the number of training epochs increases. A significant reduction in data misfit is observed at the end of the first epoch, indicating that the training of the network converges quickly. The gradual increase in model roughness shown in Figure 3B suggests that the well-trained PGAE possesses the capability to produce models with complex structures. After 20 epochs, the loss function, data misfit, and model roughness exhibit minimal changes, indicating the completion of the training stage. The network was trained using an NVIDIA Tesla K80 GPU with 12 GB of onboard memory, taking approximately 40 min.

FIGURE 3

3.2 Synthetic examples

To evaluate the performance of the trained PGAE, we simulate numerous synthetic 1D MT data as test samples, which share the same observation frequencies as the training samples. For comparative analysis, we also invert these synthetic data using the classical Occam’s inversion (). To further assess the noise adaptability of the trained PGAE, we introduce 5% Gaussian noise into the test samples. As illustrated in Figures 4, 5, the well-trained PGAE produces models comparable to those generated by the Occam’s inversion, with both methods yielding a satisfactory data fit.

FIGURE 4

FIGURE 5

Although no noise was intentionally added to the training samples, the network demonstrates remarkable adaptability to noise. Given that noise is unavoidable in real-world data, we further evaluate the noise tolerance of the trained PGAE by testing it against three higher levels of Gaussian noise: 10%, 20%, and 30%. As illustrated in Figures 6, 7, the models predicted by the PGAE are comparable to those derived from the Occam’s inversion, indicating that the PGAE exhibits significant noise tolerance even at high levels of Gaussian noise.

FIGURE 6

FIGURE 7

3.3 Real-world examples

We select 11 MT sites recorded in the Tarim Basin for our real-world examples (Figure 8). The phase tensor analysis () shows that the resistivity distribution in the shallow area closely resembles a 1D structure (Figure 9), suitable for 1D inversion. Data from these sites were recorded using the Phoenix MTU-V5 system over approximately 22 h during the summer of 2019 and subsequently processed with the robust MT time series processing code, EMTF (), to estimate the impedance and vertical transfer functions. We utilize 41 frequencies ranging from 515 Hz to 0.0439 Hz, which share a similar bandwidth with the frequencies of the training samples but employ distinct frequency values. This enables the application of the scaling laws described in Section 2.3.

FIGURE 8

FIGURE 9

).

As illustrated in Figure 10, we first transform the frequencies of the real-world data to align with the frequency range of the training data. Subsequently, we employ the linear interpolation algorithm to match the number of real-world data to the neurons of the input layer of the trained ANN. Once the trained ANN generates an inverted model, the final step is transforming it back to its original scale, ensuring consistency with the real-world system. In this scenario, we scale the real-world frequencies by 10000/515, resistivity by 10, and model size by , meeting the conditions specified in Equation 19. The models predicted by PGAE are comparable to those inverted by Occam’s method (Figure 11). Both methods yield excellent data fitting (Figure 12). Additionally, we present the pseudo-sections of the 1D models derived from both methods in Figure 13, revealing consistent 2D structures. The prediction stage of PGAE requires only 0.113 s for all 11 sites, significantly faster than the Occam’s inversion, which takes approximately 5 s per site.

FIGURE 10

FIGURE 11

FIGURE 12

FIGURE 13

4 Discussion and conclusion

The numerical experiments demonstrate that PGAE enhanced by scaling laws performs well, producing comparable models as the class Occam’s inversion does with higher efficiency after being well-trained. Here, we discuss its advantages and flaws in three aspects: model complexity, network architecture, and reusability of the network.

Model complexity is a formidable barrier for ANN-based MT inversions, limiting both the accuracy and generalization of these methods across diverse geological settings. The real-world resistivity models are inherently complex due to the highly heterogeneous nature of the Earth’s crust and mantle (). This complexity is further compounded by varying geological features, such as fault zones, fluid-bearing formations, and volcanic structures. In traditional inversion methods (; ), model complexity can be managed through regularization techniques and by setting constraints based on prior geological information. The supervised ANNs (; ; ), while powerful, are inherently data-driven and lack a physical basis. Addressing the challenge requires advances in hybrid modeling techniques and the integration of prior knowledge into the ANN framework to balance data-driven learning with geological realism. PGAE employs the forward operator as a decoder, which includes the physical laws behind the specific inverse problem. The framework can train networks without labels (models) and give a chance to add prior constraints into the training stage, which will significantly enhance the ability to reveal complex models. Moreover, future training datasets should include both synthetic and real-world data, which provide more real information related to the real tectonic and geological background.

Regarding the network architecture, it is not the primary focus of our study in the current stage. We focused on exploring the fundamental idea of PGAE, hence employing a straightforward and simple enough network with only one hidden layer. This simple design demonstrates a satisfactory ability to map the data space to the model space for 1D MT problems. However, the architecture may become crucial when addressing higher-dimensional inverse problems due to the significantly increased number of unknowns (). A network with an insufficient degree of freedom, meaning an inadequate number of network parameters, may struggle to train effectively on the provided samples. Consequently, it would likely exhibit poor prediction performance when applied to real-world data. Conversely, while a network endowed with an excessive degree of freedom can be easily trained to fit the training and evaluation samples, this requires considerable computational time. Additionally, it may lead to overfitting, which poses challenges in maintaining the network’s generalization capabilities (). This problem is particularly pronounced in MT inversions, where the resistivity structure changes significantly with depth and electromagnetic responses are frequency-dependent. Each frequency corresponds to different depths, necessitating the ANN to learn the relationships across a broad spectrum of frequencies accurately. Theoretically, no established mathematical principles definitively determine the optimal network architecture (; ), making trial-and-error the only practical approach to designing a network.

In previous studies (e.g., ; ), trained networks cannot be reused and must be re-trained from scratch when the observation system changes. However, our study makes an important step towards improving the reusability of well-trained ANNs in different observation systems. For 1D MT inverse problems, only variations in frequency range should be considered. By applying scaling laws, the trained ANN can be used in cases with nearly identical frequency bandwidths, expanding its applicability. When the frequency range of a new application spans different orders of magnitude, the trained ANN may lose its effectiveness. Therefore, it is essential to train separate ANNs tailored to different bandwidths to ensure reliable performance across a broader range of scenarios. For 3D problems, the complexity of the observation system increases further. Differences in site locations must be considered to ensure the reusability of trained ANNs. As site locations significantly affect the data characteristics in MT inversions, data from a new group of sites cannot be directly applied to the trained network without some transformation. Thus, data reconstruction between different site groups is a critical factor that must be addressed in future ANN-based MT inversion efforts. Techniques such as compressed sensing (), which can transform data from one set of locations to another, may offer potential solutions to this issue. Developing robust methods for data reconstruction should be a significant challenge.

In summary, we utilized PGAE along with scaling laws to improve the accuracy and generalization of the well-trained ANN for MT inversion. Specifically, PGAE takes into account the data misfit, while scaling laws enable the ANN to perform effectively even when the frequency range of an MT survey differs from the range used during the network’s original training. Fully enhancing the practicability, efficiency, and dependability of ANN-based inversions, particularly in the context of complex MT inverse problems, will require further research and innovation. Future research should focus on improving the adaptability of trained ANNs to both model complexity and observation systems, thereby increasing the utility of ANNs across diverse MT applications. Overcoming these challenges will be critical before ANN-based methods can be widely adopted in industry-level MT inversion software.

Statements

Data availability statement

The datasets presented in this study can be found in online repositories. The names of the repository/repositories and accession number(s) can be found below: https://github.com/Geo-LianLiu/MTPGAE.

Author contributions

LL: Conceptualization, Formal Analysis, Investigation, Methodology, Software, Validation, Writing–original draft. BY: Funding acquisition, Resources, Supervision, Validation, Writing–review and editing. YZ: Conceptualization, Investigation, Methodology, Validation, Writing–review and editing.

Funding

The author(s) declare that financial support was received for the research, authorship, and/or publication of this article. This research was financially supported by NSFC (grant 42474103).

Acknowledgments

We thank the editors and reviewers for their helpful suggestions.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Generative AI statement

The author(s) declare that no Generative AI was used in the creation of this manuscript.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

Summary

Keywords

inverse problem, magnetotellurics, artificial neural network, generalization, scaling laws

Citation

Liu L, Yang B and Zhang Y (2024) Inverting magnetotelluric data using a physics-guided auto-encoder with scaling laws extension. Front. Earth Sci. 12:1510962. doi: 10.3389/feart.2024.1510962

Received

14 October 2024

Accepted

28 November 2024

Published

16 December 2024

Volume

12 - 2024

Edited by

Maxim Smirnov, Luleå University of Technology, Sweden

Reviewed by

Jinghe Li, Guilin University of Technology, China

Rui Guo, Tsinghua University, China

Updates

Copyright

*Correspondence: Bo Yang,

Disclaimer

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.

Outline

Figures

Cite article

Copy to clipboard


Export citation file


Share article

Article metrics