Random noise attenuation is a critical step in seismic data processing. Since the distribution of field noise is complex and unknown, this poses a challenge to noise attenuation methods where the default noise distribution is known, such as a Gaussian distribution. To address this issue, we propose a method called seismic random noise Gaussianization framework (SRNGF) to attenuate random noise with unknown distribution. Specifically, SRNGF couples the Gaussianization subproblem and Gaussian denoising subproblem based on the plug-and-play (PaP) framework. The Gaussianization submodule with learnable parameters maps seismic data with the noise of unknown distribution to the one corrupted by Gaussian noise. The learnable parameters can be updated through unsupervised online training according to the noise of the unknown distribution on a single field data. The Gaussian denoising submodule, which can be replaced by seismic denoiser, aims only for Gaussian noise removal, making SRNGF incorporate the denoiser prior to seismic data. Thus, we incorporate different kinds of seismic [nondeep/deep learning (DL)] denoisers into SRNGF and give their corresponding implementations of SRNGF. Experiments on noise with unknown distribution qualitatively and quantitatively validate the superiority of SRNGF. SRNGF improves the results of non-DL/DL seismic denoiser by a large margin.
Basement relief gravimetry is crucial in geophysics, especially for oil exploration and mineral prospecting. It involves solving an inverse problem to infer geological model parameters from observed data. The model represents basement relief with constant-density prisms, and the data reflect gravitational anomalies from these prisms. Inverse problems are often ill-posed, meaning small data changes can lead to large solution variations. To mitigate this, regularization techniques like Tikhonov's are used to stabilize solutions. This study compares regularization methods applied to gravimetric inversion, including Smoothness Constraints, Total Variation, Discrete Cosine Transform (DCT), and Discrete Wavelet Transform (DWT) using Daubechies D4 wavelets. Optimization, particularly with Genetic Algorithms (GA), is used to find prism depths that best match observed anomalies. GA, inspired by natural selection, selects the best solutions to minimize the objective function. The results, evaluated through fit metrics and error analysis, show the effectiveness of all regularization methods and GA, with the Smoothness constraint performing best in synthetic models. For the real data model, all methods performed similarly.
Physics-Informed Neural Networks (PINNs) have emerged as a key tool in Scientific Machine Learning since their introduction in 2017, enabling the efficient solution of ordinary and partial differential equations using sparse measurements. Over the past few years, significant advancements have been made in the training and optimization of PINNs, covering aspects such as network architectures, adaptive refinement, domain decomposition, and the use of adaptive weights and activation functions. A notable recent development is the Physics-Informed Kolmogorov-Arnold Networks (PIKANS), which leverage a representation model originally proposed by Kolmogorov in 1957, offering a promising alternative to traditional PINNs. In this review, we provide a comprehensive overview of the latest advancements in PINNs, focusing on improvements in network design, feature expansion, optimization techniques, uncertainty quantification, and theoretical insights. We also survey key applications across a range of fields, including biomedicine, fluid and solid mechanics, geophysics, dynamical systems, heat transfer, chemical engineering, and beyond. Finally, we review computational frameworks and software tools developed by both academia and industry to support PINN research and applications.
The recent development of deep learning (DL) methods for computer vision has been driven by the creation of open benchmark datasets on which new algorithms can be tested and compared with reproducible results. Although DL methods have many applications in geophysics, few real seismic datasets are available for benchmarking DL models, especially for denoising real data, which is one of the main problems in seismic data processing scenarios in the oil and gas industry. This article presents a benchmark dataset composed of synthetic seismic data corrupted with noise extracted from a filtering process implemented on real data. In this work, a comparison between two well-known DL-based denoising models is conducted on this dataset, which is proposed as a benchmark for accelerating the development of new solutions for seismic data denoising. This work also introduces a new evaluation metric that can capture small variations in model results. The results show that DL models are effective at denoising seismic data, but some issues remain to be solved.
Full waveform inversion (FWI) plays a crucial role in the field of geophysics. There has been lots of research about applying deep learning (DL) methods to FWI. The success of DL-FWI relies significantly on the quantity and diversity of the datasets. Nevertheless, existing FWI datasets, like OpenFWI, where sources have fixed locations or identical frequencies, provide limited information and do not represent the complex real-world scene. For instance, low frequencies help in resolving larger-scale structures. High frequencies allow for a more detailed subsurface features. %A single source frequency is insufficient to describe subsurface structural properties. We consider that simultaneously using sources with different frequencies, instead of performing inversion using low frequencies data and then gradually introducing higher frequencies data, has rationale and potential advantages. Hence, we develop three enhanced datasets based on OpenFWI where each source have varying locations, frequencies or both. Moreover, we propose a novel deep operator network (DeepONet) architecture Inversion-DeepONet for FWI. We utilize convolutional neural network (CNN) to extract the features from seismic data in branch net. Source parameters, such as locations and frequencies, are fed to trunk net. Then another CNN is employed as the decoder of DeepONet to reconstruct the velocity models more effectively. Through experiments, we confirm the superior performance on accuracy and generalization ability of our network, compared with existing data-driven FWI methods.
Identifying directed spectral information flow between multivariate time series is important for many applications in finance, climate, geophysics and neuroscience. Spectral Granger causality (SGC) is a prediction-based measure characterizing directed information flow at specific oscillatory frequencies. However, traditional vector autoregressive (VAR) approaches are insufficient to assess SGC when time series have mixed frequencies (MF) or are coupled by nonlinearity. Here we propose a time-frequency canonical correlation analysis approach ("MF-TFCCA") to assess the strength and driving frequency of spectral information flow. We validate the approach with extensive computer simulations on MF time series under various interaction conditions and further assess statistical significance of the estimate with surrogate data. In various benchmark comparisons, MF-TFCCA consistently outperforms the traditional parametric MF-VAR model in both computational efficiency and detection accuracy, and recovers the dominant driving frequencies. We further apply MF-TFCCA to real-life finance, climate and neuroscience data. Our analysis framework provides an exploratory and computationally efficient nonparametric approach to quantify directed information flow between MF time series in the presence of complex and nonlinear interactions.
Random noise attenuation (NA) is a key step in seismic field data processing. With the rise of artificial intelligence, deep learning (DL) algorithms are gradually introduced into seismic random noise suppression. The most commonly used DL paradigm takes mean squared error (MSE) as loss function, the default assumption is that its error term obeys independently identically distribution (IID) Gaussian, and its noise level involved preset-hyperparameters in the local area of data cannot be adjusted adaptively in the training phase. This leads to the poor generalization of deep denoiser on Non-IID noises. In this study, we propose a DL framework based on Non-IID pixel-wise Gaussian noise modeling, which integrates NA and noise level estimation into a unique Bayesian framework. The framework can adaptively characterize the noise and data distribution in the local area of noisy data through the variational inference (VI) technique, which allows the network to see more noises of varying degrees and learn effective information from them. Thus, our proposed framework called VI-Non-IID inclines to have better noise characterization and generalization capabilities, which brings better performance on seismic field NA. Furthermore, we conduct a series of experiments on seismic synthetic and field data to test the performance of two implementations of VI-NonIID: VI-Non-IID (Unet) and VI-Non-IID denoising convolution network (DnCNN). A lot of results validate the superiority of our proposed VI-Non-IID framework. Specifically, VI-Non-IID can explicitly predict the denoised data and its corresponding noise level map simultaneously, and succeed in attenuating unknown field noises while preserving the useful seismic signals.
Usually, inverse problems are ill-posed. The solution to the inverse problem is indeterminate, meaning that for given observational data, there may be multiple possible solutions. It is not sufficient to give a definite solution to common seismic inverse problems. In this study, we provide stochastic solutions for seismic inverse problems (denoising and reconstruction). We sample a range of possible and high -quality solutions for a given observation with various degradations from the posterior distribution through Langevin dynamics with conditional score function, all shown to be reasonable results; for example, the stochastic solutions we sampled may contain as many geological structures of interest to the expert as possible. Experimental results on synthetic and field data verify the superiority of posterior sampling. In particular, our method has obvious advantages over other methods, such as traditional and (supervised, self-supervised, and unsupervised) deep learning (DL) methods, especially in denoising under extremely low signal-to-noise ratio (SNR) and reconstruction for data with consecutively missing traces and noise. We also analyze the advantages of our approach and concluded that successful generative modeling of seismic data by the score-based generative models (SGMs) is the key to posterior sampling for the inverse problems, which all benefit from the seismic data prior implicit in the trained score network in the SGMs.
The availability of seismic data has specific implications for inversion and interpretation methods, especially for learning-based methods such as deep learning. Forward modeling or acquisition of field data are common means of obtaining seismic data, which can be costly, time-consuming and labor-intensive. This paper provides a generative modeling method for generating seismic data. We use a score-based generative model to learn the distribution of target seismic data set. Specifically, we learn the gradient of the target data set distribution through score matching using the noise conditional score network (NCSN). Then, we sample high-quality samples similar to the target data set through Langevin dynamics with learned NCSN. We take the seismic records synthesized by Marmousi as an example to show the powerful generative modeling capabilities of the generative model. By sampling from a prior distribution (Gaussian distribution), the generative model can generate diverse samples and have good interpretability. For example, by interpolating two random data points from the prior distribution, the generated data has manifold continuity in certain features (such as amplitude, inclination, polarity, number of events, etc.).
The availability of data is limited in some fields, especially for object detection tasks, where it is necessary to have correctly labeled bounding boxes around each object. A notable example of such data scarcity is found in the domain of marine biology, where it is useful to develop methods to automatically detect submarine species for environmental monitoring. To address this data limitation, the state-of-the-art machine learning strategies employ two main approaches. The first involves pretraining models on existing datasets before generalizing to the specific domain of interest. The second strategy is to create synthetic datasets specifically tailored to the target domain using methods like copy-paste techniques or ad-hoc simulators. The first strategy often faces a significant domain shift, while the second demands custom solutions crafted for the specific task. In response to these challenges, here we propose a transfer learning framework that is valid for a generic scenario. In this framework, generated images help to improve the performances of an object detector in a few-real data regime. This is achieved through a diffusion-based generative model that was pretrained on large generic datasets. With respect to the state-of-the-art, we find that it is not necessary to fine tune the generative model on the specific domain of interest. We believe that this is an important advance because it mitigates the labor-intensive task of manual labeling the images in object detection tasks. We validate our approach focusing on fishes in an underwater environment, and on the more common domain of cars in an urban setting. Our method achieves detection performance comparable to models trained on thousands of images, using only a few hundreds of input data. Our results pave the way for new generative AI-based protocols for machine learning applications in various domains.
The solution of inverse problems is of fundamental interest in medical and astronomical imaging, geophysics as well as engineering and life sciences. Recent advances were made by using methods from machine learning, in particular deep neural networks. Most of these methods require a huge amount of (paired) data and computer capacity to train the networks, which often may not be available. Our paper addresses the issue of learning from small data sets by taking patches of very few images into account. We focus on the combination of model-based and data-driven methods by approximating just the image prior, also known as regularizer in the variational model. We review two methodically different approaches, namely optimizing the maximum log-likelihood of the patch distribution, and penalizing Wasserstein-like discrepancies of whole empirical patch distributions. From the point of view of Bayesian inverse problems, we show how we can achieve uncertainty quantification by approximating the posterior using Langevin Monte Carlo methods. We demonstrate the power of the methods in computed tomography, image super-resolution, and inpainting. Indeed, the approach provides also high-quality results in zero-shot super-resolution, where only a low-resolution image is available. The paper is accompanied by a GitHub repository containing implementations of all methods as well as data examples so that the reader can get their own insight into the performance.
Direct numerical simulations (DNS) are accurate but computationally expensive for predicting materials evolution across timescales, due to the complexity of the underlying evolution equations, the nature of multiscale spatio-temporal interactions, and the need to reach long-time integration. We develop a new method that blends numerical solvers with neural operators to accelerate such simulations. This methodology is based on the integration of a community numerical solver with a U-Net neural operator, enhanced by a temporal-conditioning mechanism that enables accurate extrapolation and efficient time-to-solution predictions of the dynamics. We demonstrate the effectiveness of this framework on simulations of microstructure evolution during physical vapor deposition modeled via the phase-field method. Such simulations exhibit high spatial gradients due to the co-evolution of different material phases with simultaneous slow and fast materials dynamics. We establish accurate extrapolation of the coupled solver with up to 16.5$\times$ speed-up compared to DNS. This methodology is generalizable to a broad range of evolutionary models, from solid mechanics, to fluid dynamics, geophysics, climate, and more.
This paper introduces an approach to enhance seismic fault recognition through self-supervised pretraining. Seismic fault interpretation holds great significance in the fields of geophysics and geology. However, conventional methods for seismic fault recognition encounter various issues, including dependence on data quality and quantity, as well as susceptibility to interpreter subjectivity. Currently, automated fault recognition methods proposed based on small synthetic datasets experience performance degradation when applied to actual seismic data. To address these challenges, we have introduced the concept of self-supervised learning, utilizing a substantial amount of relatively easily obtainable unlabeled seismic data for pretraining. Specifically, we have employed the Swin Transformer model as the core network and employed the SimMIM pretraining task to capture unique features related to discontinuities in seismic data. During the fine-tuning phase, inspired by edge detection techniques, we have also refined the structure of the Swin-UNETR model, enabling multiscale decoding and fusion for more effective fault detection. Experimental results demonstrate that our proposed method attains state-of-the-art performance on the Thebe dataset, as measured by the OIS and ODS metrics.
Seismic full waveform inversion (FWI) is a widely used technique in geophysics for inferring subsurface structures from seismic data. And InversionNet is one of the most successful data-driven machine learning models that is applied to seismic FWI. However, the high computing costs to run InversionNet have made it challenging to be efficiently deployed on edge devices that are usually resource-constrained. Therefore, we propose to employ the structured pruning algorithm to get a lightweight version of InversionNet, which can make an efficient inference on edge devices. And we also made a prototype with Raspberry Pi to run the lightweight InversionNet. Experimental results show that the pruned InversionNet can achieve up to 98.2 % reduction in computing resources with moderate model performance degradation.
The Gravity Recovery and Climate Experiment (GRACE) satellite mission, spanning from 2002 to 2017, has provided a valuable dataset for monitoring variations in Earth's gravity field, enabling diverse applications in geophysics and hydrology. The mission was followed by GRACE Follow-On in 2018, continuing data collection efforts. The monthly Earth gravity field, derived from the integration different instruments onboard satellites, has shown inconsistencies due to various factors, including gaps in observations for certain instruments since the beginning of the GRACE mission. With over two decades of GRACE and GRACE Follow-On data now available, this paper proposes an approach to fill the data gaps and forecast GRACE accelerometer data. Specifically, we focus on accelerometer data and employ Long Short-Term Memory (LSTM) networks to train a model capable of predicting accelerometer data for all three axes. In this study, we describe the methodology used to preprocess the accelerometer data, prepare it for LSTM training, and evaluate the model's performance. Through experimentation and validation, we assess the model's accuracy and its ability to predict accelerometer data for the three axes. Our results demonstrate the effectiveness of the LSTM forecasting model in filling gaps and forecasting GRACE accelerometer data.
Scientific Machine Learning (SciML) has advanced recently across many different areas in computational science and engineering. The objective is to integrate data and physics seamlessly without the need of employing elaborate and computationally taxing data assimilation schemes. However, preprocessing, problem formulation, code generation, postprocessing and analysis are still time consuming and may prevent SciML from wide applicability in industrial applications and in digital twin frameworks. Here, we integrate the various stages of SciML under the umbrella of ChatGPT, to formulate MyCrunchGPT, which plays the role of a conductor orchestrating the entire workflow of SciML based on simple prompts by the user. Specifically, we present two examples that demonstrate the potential use of MyCrunchGPT in optimizing airfoils in aerodynamics, and in obtaining flow fields in various geometries in interactive mode, with emphasis on the validation stage. To demonstrate the flow of the MyCrunchGPT, and create an infrastructure that can facilitate a broader vision, we built a webapp based guided user interface, that includes options for a comprehensive summary report. The overall objective is to extend MyCrunchGPT to handle diverse problems in computational mechanics, design, optimization and controls, and general scientific computing tasks involved in SciML, hence using it as a research assistant tool but also as an educational tool. While here the examples focus in fluid mechanics, future versions will target solid mechanics and materials science, geophysics, systems biology and bioinformatics.
Subsurface imaging involves solving full waveform inversion (FWI) to predict geophysical properties from measurements. This problem can be reframed as an image-to-image translation, with the usual approach being to train an encoder-decoder network using paired data from two domains: geophysical property and measurement. A recent seminal work (InvLINT) demonstrates there is only a linear mapping between the latent spaces of the two domains, and the decoder requires paired data for training. This paper extends this direction by demonstrating that only linear mapping necessitates paired data, while both the encoder and decoder can be learned from their respective domains through self-supervised learning. This unveils an intriguing phenomenon (named Auto-Linear) where the self-learned features of two separate domains are automatically linearly correlated. Compared with existing methods, our Auto-Linear has four advantages: (a) solving both forward and inverse modeling simultaneously, (b) applicable to different subsurface imaging tasks and achieving markedly better results than previous methods, (c)enhanced performance, especially in scenarios with limited paired data and in the presence of noisy data, and (d) strong generalization ability of the trained encoder and decoder.
We present the Seismic Laboratory for Imaging and Modeling/Monitoring (SLIM) open-source software framework for computational geophysics and, more generally, inverse problems involving the wave-equation (e.g., seismic and medical ultrasound), regularization with learned priors, and learned neural surrogates for multiphase flow simulations. By integrating multiple layers of abstraction, our software is designed to be both readable and scalable. This allows researchers to easily formulate their problems in an abstract fashion while exploiting the latest developments in high-performance computing. We illustrate and demonstrate our design principles and their benefits by means of building a scalable prototype for permeability inversion from time-lapse crosswell seismic data, which aside from coupling of wave physics and multiphase flow, involves machine learning.
Hydrocarbon prospect risking is a critical application in geophysics predicting well outcomes from a variety of data including geological, geophysical, and other information modalities. Traditional routines require interpreters to go through a long process to arrive at the probability of success of specific outcomes. AI has the capability to automate the process but its adoption has been limited thus far owing to a lack of transparency in the way complicated, black box models generate decisions. We demonstrate how LIME -- a model-agnostic explanation technique -- can be used to inject trust in model decisions by uncovering the model's reasoning process for individual predictions. It generates these explanations by fitting interpretable models in the local neighborhood of specific datapoints being queried. On a dataset of well outcomes and corresponding geophysical attribute data, we show how LIME can induce trust in model's decisions by revealing the decision-making process to be aligned to domain knowledge. Further, it has the potential to debug mispredictions made due to anomalous patterns in the data or faulty training datasets.
Inverse source problems are central to many applications in acoustics, geophysics, non-destructive testing, and more. Traditional imaging methods suffer from the resolution limit, preventing distinction of sources separated by less than the emitted wavelength. In this work we propose a method based on physically-informed neural-networks for solving the source refocusing problem, constructing a novel loss term which promotes super-resolving capabilities of the network and is based on the physics of wave propagation. We demonstrate the approach in the setup of imaging an a-priori unknown number of point sources in a two-dimensional rectangular waveguide from measurements of wavefield recordings along a vertical cross-section. The results show the ability of the method to approximate the locations of sources with high accuracy, even when placed close to each other.