PyTorchKR Papers

최신 논문

총 187편
태그·기간 등 상세 필터

카테고리 탐색

로그인 후 이 태그 구독

arXiv:2110.09813공식 코드 · Papers with Code

Multi-Objective Loss Balancing for Physics-Informed Deep Learning

Physics-Informed Neural Networks (PINN) are algorithms from deep learning leveraging physical laws by including partial differential equations together with a respective set of boundary and initial conditions as penalty terms into their loss function. In this work, we observe the significant role of correctly weighting the combination of multiple competitive loss functions for training PINNs effectively. To this end, we implement and evaluate different methods aiming at balancing the contributions of multiple terms of the PINNs loss function and their gradients. After reviewing of three existing loss scaling approaches (Learning Rate Annealing, GradNorm and SoftAdapt), we propose a novel self-adaptive loss balancing scheme for PINNs named \emph{ReLoBRaLo} (Relative Loss Balancing with Random Lookback). We extensively evaluate the performance of the aforementioned balancing schemes by solving both forward as well as inverse problems on three benchmark PDEs for PINNs: Burgers' equation, Kirchhoff's plate bending equation and Helmholtz's equation. The results show that ReLoBRaLo is able to consistently outperform the baseline of existing scaling methods in terms of accuracy, while also inducing significantly less computational overhead.

arXiv:2109.10793

Physics-informed Neural Networks-based Model Predictive Control for Multi-link Manipulators

We discuss nonlinear model predictive control (NMPC) for multi-body dynamics via physics-informed machine learning methods. Physics-informed neural networks (PINNs) are a promising tool to approximate (partial) differential equations. PINNs are not suited for control tasks in their original form since they are not designed to handle variable control actions or variable initial values. We thus present the idea of enhancing PINNs by adding control actions and initial conditions as additional network inputs. The high-dimensional input space is subsequently reduced via a sampling strategy and a zero-hold assumption. This strategy enables the controller design based on a PINN as an approximation of the underlying system dynamics. The additional benefit is that the sensitivities are easily computed via automatic differentiation, thus leading to efficient gradient-based algorithms. Finally, we present our results using our PINN-based MPC to solve a tracking problem for a complex mechanical system, a multi-link manipulator.

arXiv:2109.03956

AdjointNet: Constraining machine learning models with physics-based codes

Physics-informed Machine Learning has recently become attractive for learning physical parameters and features from simulation and observation data. However, most existing methods do not ensure that the physics, such as balance laws (e.g., mass, momentum, energy conservation), are constrained. Some recent works (e.g., physics-informed neural networks) softly enforce physics constraints by including partial differential equation (PDE)-based loss functions but need re-discretization of the PDEs using auto-differentiation. Training these neural nets on observational data showed that one could solve forward and inverse problems in one shot. They evaluate the state variables and the parameters in a PDE. This re-discretization of PDEs is not necessarily an attractive option for domain scientists that work with physics-based codes that have been developed for decades with sophisticated discretization techniques to solve complex process models and advanced equations of state. This paper proposes a physics constrained machine learning framework, AdjointNet, allowing domain scientists to embed their physics code in neural network training workflows. This embedding ensures that physics is constrained everywhere in the domain. Additionally, the mathematical properties such as consistency, stability, and convergence vital to the numerical solution of a PDE are still satisfied. We show that the proposed AdjointNet framework can be used for parameter estimation (and uncertainty quantification by extension) and experimental design using active learning. The applicability of our framework is demonstrated for four flow cases. Results show that AdjointNet-based inversion can estimate process model parameters with reasonable accuracy. These examples demonstrate the applicability of using existing software with no changes in source code to perform accurate and reliable inversion of model parameters.

arXiv:2108.00037공식 코드 · Papers with Code

Physics-Informed Machine Learning Method for Large-Scale Data Assimilation Problems

We develop a physics-informed machine learning approach for large-scale data assimilation and parameter estimation and apply it for estimating transmissivity and hydraulic head in the two-dimensional steady-state subsurface flow model of the Hanford Site given synthetic measurements of said variables. In our approach, we extend the physics-informed conditional Karhunen-Lo\'{e}ve expansion (PICKLE) method for modeling subsurface flow with unknown flux (Neumann) and varying head (Dirichlet) boundary conditions. We demonstrate that the PICKLE method is comparable in accuracy with the standard maximum a posteriori (MAP) method, but is significantly faster than MAP for large-scale problems. Both methods use a mesh to discretize the computational domain. In MAP, the parameters and states are discretized on the mesh; therefore, the size of the MAP parameter estimation problem directly depends on the mesh size. In PICKLE, the mesh is used to evaluate the residuals of the governing equation, while the parameters and states are approximated by the truncated conditional Karhunen-Lo\'{e}ve expansions with the number of parameters controlled by the smoothness of the parameter and state fields, and not by the mesh size. For a considered example, we demonstrate that the computational cost of PICKLE increases near linearly (as $N_{FV}^{1.15}$) with the number of grid points $N_{FV}$, while that of MAP increases much faster as $N_{FV}^{3.28}$. We demonstrated that once trained for one set of Dirichlet boundary conditions (i.e., one river stage), the PICKLE method provides accurate estimates of the hydraulic head for any value of the Dirichlet boundary conditions (i.e., for any river stage).

arXiv:2105.13813

Grey-box models for wave loading prediction

The quantification of wave loading on offshore structures and components is a crucial element in the assessment of their useful remaining life. In many applications the well-known Morison's equation is employed to estimate the forcing from waves with assumed particle velocities and accelerations. This paper develops a grey-box modelling approach to improve the predictions of the force on structural members. A grey-box model intends to exploit the enhanced predictive capabilities of data-based modelling whilst retaining physical insight into the behaviour of the system; in the context of the work carried out here, this can be considered as physics-informed machine learning. There are a number of possible approaches to establish a grey-box model. This paper demonstrates two means of combining physics (white box) and data-based (black box) components; one where the model is a simple summation of the two components, the second where the white-box prediction is fed into the black box as an additional input. Here Morison's equation is used as the physics-based component in combination with a data-based Gaussian process NARX - a dynamic variant of the more well-known Gaussian process regression. Two key challenges with employing the GP-NARX formulation that are addressed here are the selection of appropriate lag terms and the proper treatment of uncertainty propagation within the dynamic GP. The best performing grey-box model, the residual modelling GP-NARX, was able to achieve a 29.13\% and 5.48\% relative reduction in NMSE over Morison's Equation and a black-box GP-NARX respectively, alongside significant benefits in extrapolative capabilities of the model, in circumstances of low dataset coverage.

arXiv:2105.02079

Numerical Gaussian process Kalman filtering for spatiotemporal systems

We present a novel Kalman filter for spatiotemporal systems called the numerical Gaussian process Kalman filter (GPKF). Numerical Gaussian processes have recently been introduced as a physics informed machine learning method for simulating time-dependent partial differential equations without the need for spatial discretization. We bring numerical GPs into probabilistic state space form. This model is linear and its states are Gaussian distributed. These properties enable us to embed the numerical GP state space model into the recursive Kalman filter algorithm. We showcase the method using two case studies.

arXiv:2105.00075

Applying physics-based loss functions to neural networks for improved generalizability in mechanics problems

Physics-Informed Machine Learning (PIML) has gained momentum in the last 5 years with scientists and researchers aiming to utilize the benefits afforded by advances in machine learning, particularly in deep learning. With large scientific data sets with rich spatio-temporal data and high-performance computing providing large amounts of data to be inferred and interpreted, the task of PIML is to ensure that these predictions, categorizations, and inferences are enforced by, and conform to the limits imposed by physical laws. In this work a new approach to utilizing PIML is discussed that deals with the use of physics-based loss functions. While typical usage of physical equations in the loss function requires complex layers of derivatives and other functions to ensure that the known governing equation is satisfied, here we show that a similar level of enforcement can be found by implementing more simpler loss functions on specific kinds of output data. The generalizability that this approach affords is shown using examples of simple mechanical models that can be thought of as sufficiently simplified surrogate models for a wide class of problems.

arXiv:2104.13471

Discovering nonlinear resonances through physics-informed machine learning

For an ensemble of nonlinear systems that model, for instance, molecules or photonic systems, we propose a method that finds efficiently the configuration that has prescribed transfer properties. Specifically, we use physics-informed machine-learning (PIML) techniques to find the parameters for the efficient transfer of an electron (or photon) to a targeted state in a non-linear dimer. We create a machine learning model containing two variables, $\chi_D$, and $\chi_A$, representing the non-linear terms in the donor and acceptor target system states. We then introduce a data-free physics-informed loss function as $1.0 - P_j$, where $P_j$ is the probability, the electron being in the targeted state, $j$. By minimizing the loss function, we maximize the occupation probability to the targeted state. The method recovers known results in the Targeted Energy Transfer (TET) model, and it is then applied to a more complex system with an additional intermediate state. In this trimer configuration, the PIML approach discovers desired resonant paths from the donor to acceptor units. The proposed PIML method is general and may be used in the chemical design of molecular complexes or engineering design of quantum or photonic systems.

arXiv:2104.11009

Enhancing predictive skills in physically-consistent way: Physics Informed Machine Learning for Hydrological Processes

Current modeling approaches for hydrological modeling often rely on either physics-based or data-science methods, including Machine Learning (ML) algorithms. While physics-based models tend to rigid structure resulting in unrealistic parameter values in certain instances, ML algorithms establish the input-output relationship while ignoring the constraints imposed by well-known physical processes. While there is a notion that the physics model enables better process understanding and ML algorithms exhibit better predictive skills, scientific knowledge that does not add to predictive ability may be deceptive. Hence, there is a need for a hybrid modeling approach to couple ML algorithms and physics-based models in a synergistic manner. Here we develop a Physics Informed Machine Learning (PIML) model that combines the process understanding of conceptual hydrological model with predictive abilities of state-of-the-art ML models. We apply the proposed model to predict the monthly time series of the target (streamflow) and intermediate variables (actual evapotranspiration) in the Narmada river basin in India. Our results show the capability of the PIML model to outperform a purely conceptual model ($abcd$ model) and ML algorithms while ensuring the physical consistency in outputs validated through water balance analysis. The systematic approach for combining conceptual model structure with ML algorithms could be used to improve the predictive accuracy of crucial hydrological processes important for flood risk assessment.

arXiv:2101.05605

A Physics-Informed Machine Learning Model for Porosity Analysis in Laser Powder Bed Fusion Additive Manufacturing

To control part quality, it is critical to analyze pore generation mechanisms, laying theoretical foundation for future porosity control. Current porosity analysis models use machine setting parameters, such as laser angle and part pose. However, these setting-based models are machine dependent, hence they often do not transfer to analysis of porosity for a different machine. To address the first problem, a physics-informed, data-driven model (PIM), which instead of directly using machine setting parameters to predict porosity levels of printed parts, it first interprets machine settings into physical effects, such as laser energy density and laser radiation pressure. Then, these physical, machine independent effects are used to predict porosity levels according to pass, flag, fail categories instead of focusing on quantitative pore size prediction. With six learning methods evaluation, PIM proved to achieve good performances with prediction error of 10$\sim$26%. Finally, pore-encouraging influence and pore-suppressing influence were analyzed for quality analysis.

arXiv:2012.06825공식 코드 · Papers with Code

Physics-Informed Machine Learning Simulator for Wildfire Propagation

The aim of this work is to evaluate the feasibility of re-implementing some key parts of the widely used Weather Research and Forecasting WRF-SFIRE simulator by replacing its core differential equations numerical solvers with state-of-the-art physics-informed machine learning techniques to solve ODEs and PDEs, in order to transform it into a real-time simulator for wildfire spread prediction. The main programming language used is Julia, a compiled language which offers better perfomance than interpreted ones, providing Just in Time (JIT) compilation with different optimization levels. Moreover, Julia is particularly well suited for numerical computation and for the solution of complex physical models, both considering the syntax and the presence of some specific libraries such as DifferentialEquations.jl and ModellingToolkit.jl.

arXiv:2006.13380

Physics-informed machine learning for sensor fault detection with flight test data

We develop data-driven algorithms to fully automate sensor fault detection in systems governed by underlying physics. The proposed machine learning method uses a time series of typical behavior to approximate the evolution of measurements of interest by a linear time-invariant system. Given additional data from related sensors, a Kalman observer is used to maintain a separate real-time estimate of the measurement of interest. Sustained deviation between the measurements and the estimate is used to detect anomalous behavior. A decision tree, informed by integrating other sensor measurement values, is used to determine the amount of deviation required to identify a sensor fault. We validate the method by applying it to three test systems exhibiting various types of sensor faults: commercial flight test data, an unsteady aerodynamics model with dynamic stall, and a model for longitudinal flight dynamics forced by atmospheric turbulence. In the latter two cases we test fault detection for several prototypical failure modes. The combination of a learned dynamical model with the automated decision tree accurately detects sensor faults in each case.

arXiv:2011.04426

AutoMat: Accelerated Computational Electrochemical systems Discovery

Large-scale electrification is vital to addressing the climate crisis, but several scientific and technological challenges remain to fully electrify both the chemical industry and transportation. In both of these areas, new electrochemical materials will be critical, but their development currently relies heavily on human-time-intensive experimental trial and error and computationally expensive first-principles, meso-scale and continuum simulations. We present an automated workflow, AutoMat, that accelerates these computational steps by introducing both automated input generation and management of simulations across scales from first principles to continuum device modeling. Furthermore, we show how to seamlessly integrate multi-fidelity predictions such as machine learning surrogates or automated robotic experiments "in-the-loop". The automated framework is implemented with design space search techniques to dramatically accelerate the overall materials discovery pipeline by implicitly learning design features that optimize device performance across several metrics. We discuss the benefits of AutoMat using examples in electrocatalysis and energy storage and highlight lessons learned.

arXiv:2010.02011

A Physics-Informed Machine Learning Approach for Solving Heat Transfer Equation in Advanced Manufacturing and Engineering Applications

A physics-informed neural network is developed to solve conductive heat transfer partial differential equation (PDE), along with convective heat transfer PDEs as boundary conditions (BCs), in manufacturing and engineering applications where parts are heated in ovens. Since convective coefficients are typically unknown, current analysis approaches based on trial and error finite element (FE) simulations are slow. The loss function is defined based on errors to satisfy PDE, BCs and initial condition. An adaptive normalizing scheme is developed to reduce loss terms simultaneously. In addition, theory of heat transfer is used for feature engineering. The predictions for 1D and 2D cases are validated by comparing with FE results. It is shown that using engineered features, heat transfer beyond the training zone can be predicted. Trained model allows for fast evaluation of a range of BCs to develop feedback loops, realizing Industry 4.0 concept of active manufacturing control based on sensor data.

arXiv:2010.00399공식 코드 · Papers with Code

Analyzing Koopman approaches to physics-informed machine learning for long-term sea-surface temperature forecasting

Accurately predicting sea-surface temperature weeks to months into the future is an important step toward long term weather forecasting. Standard atmosphere-ocean coupled numerical models provide accurate sea-surface forecasts on the scale of a few days to a few weeks, but many important weather systems require greater foresight. In this paper we propose machine-learning approaches sea-surface temperature forecasting that are accurate on the scale of dozens of weeks. Our approach is based in Koopman operator theory, a useful tool for dynamical systems modelling. With this approach, we predict sea surface temperature in the Gulf of Mexico up to 180 days into the future based on a present image of thermal conditions and three years of historical training data. We evaluate the combination of a basic Koopman method with a convolutional autoencoder, and a newly proposed "consistent Koopman" method, in various permutations. We show that the Koopman approach consistently outperforms baselines, and we discuss the utility of our additional assumptions and methods in this sea-surface temperature domain.

arXiv:2009.05686

QRnet: optimal regulator design with LQR-augmented neural networks

In this paper we propose a new computational method for designing optimal regulators for high-dimensional nonlinear systems. The proposed approach leverages physics-informed machine learning to solve high-dimensional Hamilton-Jacobi-Bellman equations arising in optimal feedback control. Concretely, we augment linear quadratic regulators with neural networks to handle nonlinearities. We train the augmented models on data generated without discretizing the state space, enabling application to high-dimensional problems. We use the proposed method to design a candidate optimal regulator for an unstable Burgers' equation, and through this example, demonstrate improved robustness and accuracy compared to existing neural network formulations.

논문 ID: document/9158400

Physics-Informed Deep Neural Networks for Transient Electromagnetic Analysis

In this paper, we propose a deep neural network based model to predict the time evolution of field values in transient electrodynamics. The key component of our model is a recurrent neural network, which learns representations of long-term spatial-temporal dependencies in the sequence of its input data. We develop an encoder-recurrent-decoder architecture, which is trained with finite difference time domain simulations of plane wave scattering from distributed, perfect electric conducting objects. We demonstrate that, the trained network can emulate a transient electrodynamics problem with more than 17 times speed-up in simulation time compared to traditional finite difference time domain solvers.

arXiv:2008.08162

Physics-informed machine learning for the COVID-19 pandemic: Adherence to social distancing and short-term predictions for eight countries

The spread of COVID-19 during the initial phase of the first half of 2020 was curtailed to a larger or lesser extent through measures of social distancing imposed by most countries. In this work, we link directly, through machine learning techniques, infection data at a country level to a single number that signifies social distancing effectiveness. We assume that the standard SIR model gives a reasonable description of the dynamics of spreading, and thus the social distancing aspect can be modeled through time-dependent infection rates that are imposed externally. We use an exponential ansatz to analyze the SIR model, find an exact solution for the time-independent infection rate, and derive a simple first-order differential equation for the time-dependent infection rate as a function of the infected population. Using infected number data from the "first wave" of the infection from eight countries, and through physics-informed machine learning, we extract the degree of linear dependence in social distancing that led to the specific infections. We find that in the two extremes are Greece, with the highest decay slope on one side, and the US on the other with a practically flat "decay". The hierarchy of slopes is compatible with the effectiveness of the pandemic containment in each country. Finally, we train our network with data after the end of the analyzed period, and we make week-long predictions for the current phase of the infection that appear to be very close to the actual infection values.

arXiv:2008.01527

Universal Battery Performance and Degradation Model for Electric Aircraft

Development of Urban Air Mobility (UAM) concepts has been primarily focused on electric vertical takeoff and landing aircraft (eVTOLs), small aircraft which can land and takeoff vertically, and which are powered by rechargeable (typically lithium-ion) batteries. Design, analysis, and operation of eVTOLs requires fast and accurate prediction of Li-ion battery performance throughout the lifetime of the battery. eVTOL battery performance modeling must be particularly accurate at high discharge rates to ensure accurate simulation of the high power takeoff and landing portions of the flight. In this work, we generate a battery performance and thermal behavior dataset specific to eVTOL duty cycles. We use this dataset to develop a battery performance and degradation model (Cellfit) which employs physics-informed machine learning in the form of Universal Ordinary Differential Equations (U-ODE's) combined with an electrochemical cell model and degradation models which include solid electrolyte interphase (SEI) growth, lithium plating, and charge loss. We show that Cellfit with U-ODE's is better able to predict battery degradation than a mechanistic battery degradation model. We show that the improved accuracy of the degradation model improves the accuracy of the performance model. We believe that Cellfit will prove to be a valuable tool for eVTOL designers.

arXiv:2006.00544

Data-driven Optimal Power Flow: A Physics-Informed Machine Learning Approach

This paper proposes a data-driven approach for optimal power flow (OPF) based on the stacked extreme learning machine (SELM) framework. SELM has a fast training speed and does not require the time-consuming parameter tuning process compared with the deep learning algorithms. However, the direct application of SELM for OPF is not tractable due to the complicated relationship between the system operating status and the OPF solutions. To this end, a data-driven OPF regression framework is developed that decomposes the OPF model features into three stages. This not only reduces the learning complexity but also helps correct the learning bias. A sample pre-classification strategy based on active constraint identification is also developed to achieve enhanced feature attractions. Numerical results carried out on IEEE and Polish benchmark systems demonstrate that the proposed method outperforms other alternatives. It is also shown that the proposed method can be easily extended to address different test systems by adjusting only a few hyperparameters.