PyTorchKR Papers

최신 논문

총 73편
태그·기간 등 상세 필터

카테고리 탐색

로그인 후 이 태그 구독

arXiv:2007.06183

Data-driven geophysics: from dictionary learning to deep learning

Understanding the principles of geophysical phenomena is an essential and challenging task. "Model-driven" approaches have supported the development of geophysics for a long time; however, such methods suffer from the curse of dimensionality and may inaccurately model the subsurface. "Data-driven" techniques may overcome these issues with increasingly available geophysical data. In this article, we review the basic concepts of and recent advances in data-driven approaches from dictionary learning to deep learning in a variety of geophysical scenarios. Explorational geophysics including data processing, inversion and interpretation will be mainly focused. Artificial intelligence applications on geoscience involving deep Earth, earthquake, water resource, atmospheric science, satellite remoe sensing and space sciences are also reviewed. We present a coding tutorial and a summary of tips for beginners and interested geophysical readers to rapidly explore deep learning. Some promising directions are provided for future research involving deep learning in geophysics, such as unsupervised learning, transfer learning, multimodal deep learning, federated learning, uncertainty estimation, and activate learning.

arXiv:2006.13311공식 코드 · Papers with Code

70 years of machine learning in geoscience in review

This review gives an overview of the development of machine learning in geoscience. A thorough analysis of the co-developments of machine learning applications throughout the last 70 years relates the recent enthusiasm for machine learning to developments in geoscience. I explore the shift of kriging towards a mainstream machine learning method and the historic application of neural networks in geoscience, following the general trend of machine learning enthusiasm through the decades. Furthermore, this chapter explores the shift from mathematical fundamentals and knowledge in software development towards skills in model validation, applied statistics, and integrated subject matter expertise. The review is interspersed with code examples to complement the theoretical foundations and illustrate model validation and machine learning explainability for science. The scope of this review includes various shallow machine learning methods, e.g. Decision Trees, Random Forests, Support-Vector Machines, and Gaussian Processes, as well as, deep neural networks, including feed-forward neural networks, convolutional neural networks, recurrent neural networks and generative adversarial networks. Regarding geoscience, the review has a bias towards geophysics but aims to strike a balance with geochemistry, geostatistics, and geology, however excludes remote sensing, as this would exceed the scope. In general, I aim to provide context for the recent enthusiasm surrounding deep learning with respect to research, hardware, and software developments that enable successful application of shallow and deep machine learning in all disciplines of Earth science.

arXiv:2005.11106

On the suitability of generalized regression neural networks for GNSS position time series prediction for geodetic applications in geodesy and geophysics

In this paper, the generalized regression neural network is used to predict the GNSS position time series. Using the IGS 24-hour final solution data for Bad Hamburg permanent GNSS station in Germany, it is shown that the larger the training of the network, the higher the accuracy is, regardless of the time span of the time series. In order to analyze the performance of the neural network in various conditions, 14 permanent stations are used in different countries, namely, Spain, France, Romania, Poland, Russian Federation, United Kingdom, Czech Republic, Sweden, Ukraine, Italy, Finland, Slovak Republic, Cyprus, and Greece. The performance analysis is divided into two parts, continuous data-without gaps-and discontinuous ones-having intervals of gaps with no data available. Three measure of error are presented, namely, symmetric mean absolute percentage error, standard deviation, and mean of absolute errors. It is shown that for discontinuous data the position can be predicted with an accuracy of up to 6 centimeters, while the continuous data positions present a higher prediction accuracy, as high as 3 centimeters. In order to compare the results of this machine learning algorithm with the traditional statistical approaches, the Theta method is used, which is well-established for high-accuracy time series prediction. The comparison shows that the generalized regression neural network machine learning algorithm presents better accuracy than the Theta method, possibly up to 250 times. In addition, it is approximately 4.6 times faster.

arXiv:2005.02903

High-Contrast Reflection Tomography with Total-Variation Constraints

Inverse scattering is the process of estimating the spatial distribution of the scattering potential of an object by measuring the scattered wavefields around it. In this paper, we consider reflection tomography of high contrast objects that commonly occurs in ground-penetrating radar, exploration geophysics, terahertz imaging, ultrasound, and electron microscopy. Unlike conventional transmission tomography, the reflection regime is severely ill-posed since the measured wavefields contain far less spatial frequency information of the target object. We propose a constrained incremental frequency inversion framework that requires no side information from a background model of the object. Our framework solves a sequence of regularized least-squares subproblems that ensure consistency with the measured scattered wavefield while imposing total-variation and non-negativity constraints. We propose a proximal Quasi-Newton method to solve the resulting subproblem and devise an automatic parameter selection routine to determine the constraint of each subproblem. We validate the performance of our approach on synthetic low-resolution phantoms and with a mismatched forward model test on a high-resolution phantom.

arXiv:1912.01148

A Deep Convolutional Network for Seismic Shot-Gather Image Quality Classification

Deep Learning-based models such as Convolutional Neural Networks, have led to significant advancements in several areas of computing applications. Seismogram quality assurance is a relevant Geophysics task, since in the early stages of seismic processing, we are required to identify and fix noisy sail lines. In this work, we introduce a real-world seismogram quality classification dataset based on 6,613 examples, manually labeled by human experts as good, bad or ugly, according to their noise intensity. This dataset is used to train a CNN classifier for seismic shot-gathers quality prediction. In our empirical evaluation, we observe an F1-score of 93.56% in the test set.

arXiv:1912.00612

Inversion of 1D frequency- and time-domain electromagnetic data with convolutional neural networks

Inversion of electromagnetic data finds applications in many areas of geophysics. The inverse problem is commonly solved with either deterministic optimization methods (such as the nonlinear conjugate gradient or Gauss-Newton) which are prone to getting trapped in a local minimum, or probabilistic methods which are very computationally demanding. A recently emerging alternative is to employ deep neural networks for predicting subsurface model properties from measured data. This approach is entirely data-driven, does not employ traditional gradient-based techniques and provides a guess to the model instantaneously. In this study, we apply deep convolutional neural networks for 1D inversion of marine frequency-domain controlled-source electromagnetic (CSEM) data as well as onshore time-domain electromagnetic (TEM) data. Our approach yields accurate results both on synthetic and real data and provides them instantaneously. Using several networks and combining their outputs from various training epochs can also provide insights into the uncertainty distribution, which is found to be higher in the regions where resistivity anomalies are present. The proposed method opens up possibilities to estimate the subsurface resistivity distribution in exploration scenarios in real time.

arXiv:1910.02434

Distributed filtered hyperinterpolation for noisy data on the sphere

Problems in astrophysics, space weather research and geophysics usually need to analyze noisy big data on the sphere. This paper develops distributed filtered hyperinterpolation for noisy data on the sphere, which assigns the data fitting task to multiple servers to find a good approximation of the mapping of input and output data. For each server, the approximation is a filtered hyperinterpolation on the sphere by a small proportion of quadrature nodes. The distributed strategy allows parallel computing for data processing and model selection and thus reduces computational cost for each server while preserves the approximation capability compared to the filtered hyperinterpolation. We prove quantitative relation between the approximation capability of distributed filtered hyperinterpolation and the numbers of input data and servers. Numerical examples show the efficiency and accuracy of the proposed method.

arXiv:1908.03973

Enhanced Seismic Imaging with Predictive Neural Networks for Geophysics

We propose a predictive neural network architecture that can be utilized to update reference velocity models as inputs to the full waveform inversion. Deep learning models are explored to augment velocity model building workflows during processing the 3D seismic volume in salt-prone environments. Specifically, a neural network architecture, with 3D convolutional, de-convolutional layers, and 3D max-pooling, is designed to take standard amplitude 3D seismic volumes as an input. Enhanced data augmentations through generative adversarial networks and a weighted loss function enable the network to train with few sparsely annotated slices. Batch normalization is also applied for faster convergence. A 3D probability cube for salt bodies and inclusions is generated through ensembles of predictions from multiple models in order to reduce variance. Velocity models inferred from the proposed networks provide opportunities for FWI forward models to converge faster with an initial condition closer to the true model. In addition, in each iteration step, the probability cubes of salt bodies and inclusions inferred from the proposed networks can be used as a regularization term within the FWI forward modelling, which may result in an improved velocity model estimation while the output of seismic migration can be utilized as an input of the 3D neural network for subsequent iterations.

arXiv:1907.03137

Precision annealing Monte Carlo methods for statistical data assimilation and machine learning

In statistical data assimilation (SDA) and supervised machine learning (ML), we wish to transfer information from observations to a model of the processes underlying those observations. For SDA, the model consists of a set of differential equations that describe the dynamics of a physical system. For ML, the model is usually constructed using other strategies. In this paper, we develop a systematic formulation based on Monte Carlo sampling to achieve such information transfer. Following the derivation of an appropriate target distribution, we present the formulation based on the standard Metropolis-Hasting (MH) procedure and the Hamiltonian Monte Carlo (HMC) method for performing the high dimensional integrals that appear. To the extensive literature on MH and HMC, we add (1) an annealing method using a hyperparameter that governs the precision of the model to identify and explore the highest probability regions of phase space dominating those integrals, and (2) a strategy for initializing the state space search. The efficacy of the proposed formulation is demonstrated using a nonlinear dynamical model with chaotic solutions widely used in geophysics.

arXiv:1904.04445공식 코드 · Papers with Code

Semi-Supervised Segmentation of Salt Bodies in Seismic Images using an Ensemble of Convolutional Neural Networks

Seismic image analysis plays a crucial role in a wide range of industrial applications and has been receiving significant attention. One of the essential challenges of seismic imaging is detecting subsurface salt structure which is indispensable for identification of hydrocarbon reservoirs and drill path planning. Unfortunately, exact identification of large salt deposits is notoriously difficult and professional seismic imaging often requires expert human interpretation of salt bodies. Convolutional neural networks (CNNs) have been successfully applied in many fields, and several attempts have been made in the field of seismic imaging. But the high cost of manual annotations by geophysics experts and scarce publicly available labeled datasets hinder the performance of the existing CNN-based methods. In this work, we propose a semi-supervised method for segmentation (delineation) of salt bodies in seismic images which utilizes unlabeled data for multi-round self-training. To reduce error amplification during self-training we propose a scheme which uses an ensemble of CNNs. We show that our approach outperforms state-of-the-art on the TGS Salt Identification Challenge dataset and is ranked the first among the 3234 competing methods.

arXiv:1609.06500

Wavelet-Based Segmentation on the Sphere

Segmentation, a useful/powerful technique in pattern recognition, is the process of identifying object outlines within images. There are a number of efficient algorithms for segmentation in Euclidean space that depend on the variational approach and partial differential equation modelling. Wavelets have been used successfully in various problems in image processing, including segmentation, inpainting, noise removal, super-resolution image restoration, and many others. Wavelets on the sphere have been developed to solve such problems for data defined on the sphere, which arise in numerous fields such as cosmology and geophysics. In this work, we propose a wavelet-based method to segment images on the sphere, accounting for the underlying geometry of spherical data. Our method is a direct extension of the tight-frame based segmentation method used to automatically identify tube-like structures such as blood vessels in medical imaging. It is compatible with any arbitrary type of wavelet frame defined on the sphere, such as axisymmetric wavelets, directional wavelets, curvelets, and hybrid wavelet constructions. Such an approach allows the desirable properties of wavelets to be naturally inherited in the segmentation process. In particular, directional wavelets and curvelets, which were designed to efficiently capture directional signal content, provide additional advantages in segmenting images containing prominent directional and curvilinear features. We present several numerical experiments, applying our wavelet-based segmentation method, as well as the common K-means method, on real-world spherical images. These experiments demonstrate the superiority of our method and show that it is capable of segmenting different kinds of spherical images, including those with prominent directional features. Moreover, our algorithm is efficient with convergence usually within a few iterations.

arXiv:1804.03457

Universal features of mountain ridge networks on Earth

Compared to the heavily studied surface drainage systems, the mountain ridge systems have been a subject of less attention even on the empirical level, despite the fact that their structure is richer. To reduce this deficiency, we analyze different mountain ranges by means of a network approach and grasp some essential features of the ridge branching structure. We also employ a fractal analysis as it is especially suitable for describing properties of rough objects and surfaces. As our approach differs from typical analyses that are carried out in geophysics, we believe that it can initialize a research direction that will allow to shed more light on the processes that are responsible for landscape formation and will contribute to the network theory by indicating a need for the construction of new models of the network growth as no existing model can properly describe the ridge formation. We also believe that certain features of our study can offer help in the cartographic generalization. Specifically, we study structure of the ridge networks based on the empirical elevation data collected by SRTM. We consider mountain ranges from different geological periods and geographical locations. For each mountain range, we construct a simple topographic network representation (the ridge junctions are nodes) and a ridge representation (the ridges are nodes and the junctions are edges) and calculate the parameters characterizing their topology. We observe that the topographic networks inherit the fractal structure of the mountain ranges but do not show any other complex features. In contrast, the ridge networks, while lacking the proper fractality, reveal the power-law degree distributions with the exponent $1.6\le \beta \le 1.7$. By taking into account the fact that the analyzed mountains differ in many properties, these values seem to be universal for the earthly mountainous terrain.

arXiv:2503.22223v2cs.LG

DREMnet: 준공중 과도 전자기 신호를 위한 해석 가능한 노이즈 제거 프레임워크

DREMnet: An Interpretable Denoising Framework for Semi-Airborne Transient Electromagnetic Signal

준공중 과도 전자기법(SATEM)은 대규모 및 접근하기 어려운 지역에 대한 신속한 탐사를 수행할 수 있다. 그러나 획득된 신호는 종종 복잡한 노이즈에 오염되어, 후속 역산(inversion) 해석의 정확도를 저해할 수 있다. 기존 노이즈 제거 기법은 주로 매개변수 선택 전략에 의존하며, 이는 노이즈가 많은 환경에서 현장 데이터를 처리하기에 불충분하다. 심층 학습의 등장과 함께 다양한 신경망이 SATEM 신호 노이즈 제거에 활용되어 왔다. 그러나 기존 심층 학습 방법은 일반적으로 단일 매핑 학습 접근법을 사용하여 신호와 노이즈를 효과적으로 분리하는 데 어려움을 겪는다. 이러한 방법들은 부분적인 정보만 포착하며 해석 가능성이 부족하다. 이러한 한계를 극복하기 위해, 우리는 데이터를 콘텐츠(content) 및 컨텍스트(context) 요인으로 분리(disentangle)하여 복잡한 조건에서 강건하고 해석 가능한 노이즈 제거를 가능하게 하는 해석 가능한 분리 표현 학습(decoupled representation learning) 프레임워크인 DREMnet을 제안한다. CNN 및 트랜스포머 아키텍처의 한계를 해결하기 위해, 데이터 처리에 RWKV 아키텍처를 활용하고 단방향 WKV가 양방향 신호 모델링을 수행할 수 있도록 하는 Contextual-WKV 메커니즘을 도입한다. 우리가 제안한 Covering Embedding 기법은 stacked 임베딩을 통해 합성곱 신경망의 강한 국소 인식 능력을 유지한다. 테스트 데이터셋에 대한 실험 결과는 DREMnet 방법이 기존 기법들을 능가하며, 처리된 현장 데이터가 이론적 신호를 더 정확하게 반영하여 지하 전기 구조의 식별을 개선함을 보여준다.