A physics-informed variational DeepONet for predicting crack path in quasi-brittle materials

Computer Methods in Applied Mechanics and Engineering · 2022 · 365 citations

• Computer Science
• Artificial Intelligence
• Computer Science

Deep transfer operator learning for partial differential equations under conditional shift

Nature Machine Intelligence · 2022 · 112 citations

• Computer Science
• Computer Science
• Artificial Intelligence

Transfer learning enables the transfer of knowledge gained while learning to perform one task (source) to a related but different task (target), hence addressing the expense of data acquisition and labelling, potential computational power limitations and dataset distribution mismatches. We propose a new transfer learning framework for task-specific learning (functional regression in partial differential equations) under conditional shift based on the deep operator network (DeepONet). Task-specific operator learning is accomplished by fine-tuning task-specific layers of the target DeepONet using a hybrid loss function that allows for the matching of individual target samples while also preserving the global properties of the conditional distribution of the target data. Inspired by conditional embedding operator theory, we minimize the statistical distance between labelled target data and the surrogate prediction on unlabelled target data by embedding conditional distributions onto a reproducing kernel Hilbert space. We demonstrate the advantages of our approach for various transfer learning scenarios involving nonlinear partial differential equations under diverse conditions due to shifts in the geometric domain and model dynamics. Our transfer learning framework enables fast and efficient learning of heterogeneous tasks despite considerable differences between the source and target domains. A promising area for deep learning is in modelling complex physical processes described by partial differential equations (PDEs), which is computationally expensive for conventional approaches. An operator learning approach called DeepONet was recently introduced to tackle PDE-related problems, and in new work, this approach is extended with transfer learning, which transfers knowledge obtained from learning to perform one task to a related but different task.

Physics-Informed Neural Networks for Heat Transfer Problems

Journal of Heat Transfer · 2021 · 1106 citations

• Computer Science
• Artificial Intelligence
• Computer Science

Abstract Physics-informed neural networks (PINNs) have gained popularity across different engineering fields due to their effectiveness in solving realistic problems with noisy data and often partially missing physics. In PINNs, automatic differentiation is leveraged to evaluate differential operators without discretization errors, and a multitask learning problem is defined in order to simultaneously fit observed data while respecting the underlying governing laws of physics. Here, we present applications of PINNs to various prototype heat transfer problems, targeting in particular realistic conditions not readily tackled with traditional computational methods. To this end, we first consider forced and mixed convection with unknown thermal boundary conditions on the heated surfaces and aim to obtain the temperature and velocity fields everywhere in the domain, including the boundaries, given some sparse temperature measurements. We also consider the prototype Stefan problem for two-phase flow, aiming to infer the moving interface, the velocity and temperature fields everywhere as well as the different conductivities of a solid and a liquid phase, given a few temperature measurements inside the domain. Finally, we present some realistic industrial applications related to power electronics to highlight the practicality of PINNs as well as the effective use of neural networks in solving general heat transfer problems of industrial complexity. Taken together, the results presented herein demonstrate that PINNs not only can solve ill-posed problems, which are beyond the reach of traditional computational methods, but they can also bridge the gap between computational and experimental heat transfer.

Two-point stress–strain-rate correlation structure and non-local eddy viscosity in turbulent flows

Journal of Fluid Mechanics · 2021 · 42 citations

• Mechanics
• Materials science
• Physics

Abstract

A comprehensive and fair comparison of two neural operators (with practical extensions) based on FAIR data

Computer Methods in Applied Mechanics and Engineering · 2021 · 448 citations

• Computer Science
• Artificial Intelligence
• Computer Science

Neural operators can learn nonlinear mappings between function spaces and offer a new simulation paradigm for real-time prediction of complex dynamics for realistic diverse applications as well as for system identification in science and engineering. Herein, we investigate the performance of two neural operators, which have shown promising results so far, and we develop new practical extensions that will make them more accurate and robust and importantly more suitable for industrial-complexity applications. The first neural operator, DeepONet, was published in 2019 (Lu et al., 2019), and its original architecture was based on the universal approximation theorem of Chen & Chen (1995). The second one, named Fourier Neural Operator or FNO, was published in 2020, and it is based on parameterizing the integral kernel in the Fourier space. DeepONet is represented by a summation of products of neural networks (NNs), corresponding to the branch NN for the input function and the trunk NN for the output function; both NNs are general architectures, e.g., the branch NN can be replaced with a CNN or a ResNet. According to Kovachki et al. (2021), FNO in its continuous form can be viewed conceptually as a DeepONet with a specific architecture of the branch NN and a trunk NN represented by a trigonometric basis. In order to compare FNO with DeepONet computationally for realistic setups, we develop several extensions of FNO that can deal with complex geometric domains as well as mappings where the input and output function spaces are of different dimensions. We also develop an extended DeepONet with special features that provide inductive bias and accelerate training, and we present a faster implementation of DeepONet with cost comparable to the computational cost of FNO, which is based on the Fast Fourier Transform. Here we consider 16 different benchmarks to demonstrate the relative performance of the two neural operators, including instability wave analysis in hypersonic boundary layers, prediction of the vorticity field of a flapping airfoil, porous media simulations in complex-geometry domains, etc. We follow the guiding principles of FAIR (Findability, Accessibility, Interoperability, and Reusability) for scientific data management and stewardship. The performance of DeepONet and FNO is comparable for relatively simple settings, but for complex geometries the performance of FNO deteriorates greatly. We also compare theoretically the two neural operators and obtain similar error estimates for DeepONet and FNO under the same regularity assumptions.

Physics-informed machine learning

Nature Reviews Physics · 2021 · 6142 citations

• Computer Science
• Artificial Intelligence
• Machine Learning

Data-driven physics-informed constitutive metamodeling of complex fluids: A multifidelity neural network (MFNN) framework

Journal of Rheology · 2021 · 93 citations

• Computer Science
• Artificial Intelligence
• Machine Learning

In this work, we introduce a comprehensive machine-learning algorithm, namely, a multifidelity neural network (MFNN) architecture for data-driven constitutive metamodeling of complex fluids. The physics-based neural networks developed here are informed by the underlying rheological constitutive models through the synthetic generation of low-fidelity model-based data points. The performance of these rheologically informed algorithms is thoroughly investigated and compared against classical deep neural networks (DNNs). The MFNNs are found to recover the experimentally observed rheology of a multicomponent complex fluid consisting of several different colloidal particles, wormlike micelles, and other oil and aromatic particles. Moreover, the data-driven model is capable of successfully predicting the steady state shear viscosity of this fluid under a wide range of applied shear rates based on its constituting components. Building upon the demonstrated framework, we present the rheological predictions of a series of multicomponent complex fluids made by DNN and MFNN. We show that by incorporating the appropriate physical intuition into the neural network, the MFNN algorithms capture the role of experiment temperature, the salt concentration added to the mixture, as well as aging within and outside the range of training data parameters. This is made possible by leveraging an abundance of synthetic low-fidelity data that adhere to specific rheological models. In contrast, a purely data-driven DNN is consistently found to predict erroneous rheological behavior.

Non-invasive inference of thrombus material properties with physics-informed neural networks

Computer Methods in Applied Mechanics and Engineering · 2020 · 176 citations

• Computer Science
• Artificial Intelligence
• Applied mathematics

Hidden fluid mechanics: Learning velocity and pressure fields from flow visualizations

Science · 2020 · 1900 citations

• Computer Science
• Artificial Intelligence
• Computer Science

For centuries, flow visualization has been the art of making fluid motion visible in physical and biological systems. Although such flow patterns can be, in principle, described by the Navier-Stokes equations, extracting the velocity and pressure fields directly from the images is challenging. We addressed this problem by developing hidden fluid mechanics (HFM), a physics-informed deep-learning framework capable of encoding the Navier-Stokes equations into the neural networks while being agnostic to the geometry or the initial and boundary conditions. We demonstrate HFM for several physical and biomedical problems by extracting quantitative information for which direct measurements may not be possible. HFM is robust to low resolution and substantial noise in the observation data, which is important for potential applications.