基于梯度优化物理信息神经网络求解复杂非线性问题

田十方; 李彪

doi:10.7498/aps.72.20222381

摘要

近年来, 物理信息神经网络(PINNs)因其仅通过少量数据就能快速获得高精度的数据驱动解而受到越来越多的关注. 然而, 尽管该模型在部分非线性问题中有着很好的结果, 但它还是有一些不足的地方, 如它的不平衡的反向传播梯度计算导致模型训练期间梯度值剧烈振荡, 这容易导致预测精度不稳定. 基于此, 本文通过梯度统计平衡了模型训练期间损失函数中不同项之间的相互作用, 提出了一种梯度优化物理信息神经网络(GOPINNs), 该网络结构对梯度波动更具鲁棒性. 然后以Camassa-Holm (CH)方程、导数非线性薛定谔方程为例, 利用GOPINNs模拟了CH方程的peakon解和导数非线性薛定谔方程的有理波解、怪波解. 数值结果表明, GOPINNs可以有效地平滑计算过程中损失函数的梯度, 并获得了比原始PINNs精度更高的解. 总之, 本文的工作为优化神经网络的学习性能提供了新的见解, 并在求解复杂的CH方程和导数非线性薛定谔方程时用时更少, 节约了超过三分之一的时间, 并且将预测精度提高了将近10倍.

关键词:

Abstract

In recent years, physics-informed neural networks (PINNs) have attracted more and more attention for their ability to quickly obtain high-precision data-driven solutions with only a small amount of data. However, although this model has good results in some nonlinear problems, it still has some shortcomings. For example, the unbalanced back-propagation gradient calculation results in the intense oscillation of the gradient value during the model training, which is easy to lead to the instability of the prediction accuracy. Based on this, we propose a gradient-optimized physics-informed neural networks (GOPINNs) model in this paper, which proposes a new neural network structure and balances the interaction between different terms in the loss function during model training through gradient statistics, so as to make the new proposed network structure more robust to gradient fluctuations. In this paper, taking Camassa-Holm (CH) equation and DNLS equation as examples, GOPINNs is used to simulate the peakon solution of CH equation, the rational wave solution of DNLS equation and the rogue wave solution of DNLS equation. The numerical results show that the GOPINNs can effectively smooth the gradient of the loss function in the calculation process, and obtain a higher precision solution than the original PINNs. In conclusion, our work provides new insights for optimizing the learning performance of neural networks, and saves more than one third of the time in simulating the complex CH equation and the DNLS equation, and improves the prediction accuracy by nearly ten times.

Keywords:

physics-informed neural networks /
gradient optimization /
Camassa-Holm equation /
derivative nonlinear Schrödinger equation /
peakon solution /
rogue wave solution

作者及机构信息

田十方,
李彪

宁波大学数学与统计学院, 宁波　315211

通信作者: 李彪, libiao@nbu.edu.cn

基金项目: 国家自然科学基金(批准号: 12175111, 12235007)和宁波大学王宽诚幸福基金资助的课题

Authors and contacts

School of Mathematics and Statistics, Ningbo University, Ningbo 315211, China

Corresponding author: Li Biao, libiao@nbu.edu.cn

Funds: Project supported by the National Natural Science Foundation of China (Grant Nos. 12175111, 12235007) and the K. C. Wong Magna Fund in Ningbo University, China

文章全文 : translate this paragraph

参考文献

[1]	Linβ T 2001 Comput. Math. Math. Phys. 41 898 Google Scholar
[2]	Vulanović R 1988 Z. Angew. Math. Mech. 5 428 Google Scholar
[3]	Vulanović R, Nhan T A 2020 J. Comput. Appl. Math. 386 125495 Google Scholar
[4]	Gowrisankar S, Srinivasan N 2019 Appl. Math. Comput. 346 385 Google Scholar
[5]	Nie F, Wang H, Song Q, Zhao Y, Shen J, Gong M 2022 Int. J. Multiphase Flow 152 104067 Google Scholar
[6]	Lagendijk L R, Biemond J, et al. 1988 International Conference on Acoustics New York, USA, April 11–14, 1988 p60916260
[7]	Simon H 1980 Cognitive Science 4 33 Google Scholar
[8]	Busemeyer J 2015 Cognition 135 43 Google Scholar
[9]	Sharma N, Jain V, Mishra A 2018 Procedia Comput. Sci. 132 377 Google Scholar
[10]	Gu J X, Wang Z H, Jason K, Ma L y, Amir S, Shuai B, et al. 2018 Pattern Recognit. 77 354 Google Scholar
[11]	He K, Zhang X, Ren S, Sun J 2016 Las Vegas Proceedings of the IEEE Conference on Computer Vision and Pattern RecognitionLas Vegas, USA, June 27–30, 2016 p770
[12]	Dieleman S, Zen H, Simonyan K, Vinyals O, Graves A, et al 2016 arXiv: 1609.03499 [cs.SD]
[13]	Heaton J, Goodfellow I, Bengio Y, Courville A 2018 Genet Program Evolvable Mach. 19 305 Google Scholar
[14]	Alipanahi B, Delong A, Weirauch T M, Frey J B 2015 Nat. Biotechnol. 33 831 Google Scholar
[15]	Han J, Jentzen A, Weinan E 2018 Proc. Natl. Acad. Sci. 115 8505 Google Scholar
[16]	Rudy H S, Brunton L S, Proctor L J, Kutz N 2017 Sci. Adv. 3 e1602614 Google Scholar
[17]	Raissi M, Karniadakis G E 2018 J. Comput. Phys. 357 125 Google Scholar
[18]	Weinan E, Han J Q, Jentzen A 2017 Commun. Math. Stat. 5 349 Google Scholar
[19]	Sirignano J, Spiliopoulos K 2018 J. Comput. Phys. 375 1339 Google Scholar
[20]	Raissi M, Perdikaris P, Karniadakis G E 2019 J. Comput. Phys. 378 686 Google Scholar
[21]	Jagtap A D, Kharazmi E, Karniadakis G E 2020 Proc. R. Soc. A 476 20200334 Google Scholar
[22]	Revanth M, Susanta G 2021 arXiv: 2106 07606 [math.NA]
[23]	Li J, Chen Y 2020 Commun.Theor. Phys. 72 105005 Google Scholar
[24]	Li J, Chen Y 2020 Commun. Theor. Phys. 72 115003 Google Scholar
[25]	Li J, Chen Y 2021 Commun. Theor. Phys. 73 015001 Google Scholar
[26]	Pu J C, Li J, Chen Y 2021 Chin. Phys. B 30 060202 Google Scholar
[27]	Pu J C, Li J, Chen Y 2021 Nonlinear Dyn. 105 1723 Google Scholar
[28]	Pu J C, Chen Y 2022 Chaos, Solitons Fractals 160 112182 Google Scholar
[29]	Lin S N, Chen Y 2022 J. Comput. Phys. 41 898 Google Scholar
[30]	Ling L M, Mo Y F, Zeng D L 2022 Phys. Lett. A 421 127739 Google Scholar
[31]	He J S, Wang J L 2022 Phys. Lett. A 452 128432 Google Scholar
[32]	Wang L, Yan Z Y 2021 Phys. Lett. A 404 127408 Google Scholar
[33]	Wang L, Yan Z Y 2022 Phys. Lett. A 450 128373 Google Scholar
[34]	Fang Y, Wu G Z, Wang Y Y, et al. 2021 Nonlinear Dyn 105 603 Google Scholar
[35]	Zhou Z J, Yan Z Y 2021 Phys. Lett. A 387 127010 Google Scholar
[36]	Wang L, Yan Z Y 2021 Physica D 428 133037 Google Scholar
[37]	Bai Y, Chaolu T, Bilige S 2021 Nonlinear Dyn. 105 3439 Google Scholar
[38]	Wu G Z, Fang Y, Dai C Q, et al. 2021 Chaos, Solitons Fractals 152 111393 Google Scholar
[39]	Li J H, Li B 2021 Commun. Theor. Phys. 73 125001 Google Scholar
[40]	Li J H, Chen J C, Li B 2022 Nonlinear Dyn. 107 781 Google Scholar
[41]	Li J H, Li B 2022 Chaos, Solitons Fractals 164 112712 Google Scholar
[42]	Fang Y, Wu G Z, Dai C Q, et al. 2022 Chaos, Solitons Fractals 158 112118 Google Scholar
[43]	Wu G Z, Fang Y, Dai C Q, et al. 2022 Chaos, Solitons Fractals 159 112143 Google Scholar
[44]	Yuan L, Ni Y Q, Deng X Y, Hao S 2022 J. Comput. Phys. 462 111260 Google Scholar
[45]	Zeng S J, Zhang Z, Zou Q S 2022 J. Comput. Phys. 463 111232 Google Scholar
[46]	Samadi-koucheksaraee A, Ahmadianfar I, Bozorg-Haddad O, et al. 2019 Water Resour. Manage. 33 603 Google Scholar
[47]	Marcucci G, Pierangeli D, Conti C 2020 Phys. Rev. Lett. 125 093901 Google Scholar
[48]	Kingma D P, Jimmy B 2014 arXiv: 1412 6980 [cs.LG]
[49]	Glorot X, Bengio Y 2010 Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics Chia Laguna Resort, Sardinia, Italy, March 31, 2010 pp249–256
[50]	Camassa R, Holm D 1993 Phys. Rev. Lett. 71 1661 Google Scholar
[51]	Metin G, Atalay K 1998 J. Math. Phys. 39 2103 Google Scholar
[52]	Takayuki T, Miki W 1999 Phys. Lett. A 257 53 Google Scholar
[53]	Xu S W, He J S, Wang L H 2011 J. Phys. A: Math. Theor. 44 305203 Google Scholar

施引文献

图 1 GOPINNs模型

Fig. 1. GOPINNs model

下载: 全尺寸图片幻灯片

图 2 当$ c = 0.5 $时, CH方程的peakon解((8)式)的三维图

Fig. 2. Three-dimensional diagram of peakon solution (Eq. (8)) of the CH equation when $ c = 0.5 $

下载: 全尺寸图片幻灯片

图 3 (a) CH方程的peakon解((8)式); (b)应用PINNs模型模拟的预测解; (c) PINNs模型的绝对误差

Fig. 3. (a) Peakon solution of CH equation (Eq. (8)); (b) prediction solution simulated by PINNs model; (c) absolute error of PINNs model

下载: 全尺寸图片幻灯片

图 5 应用PINNs模型时, 整个迭代过程中损失函数$ {\rm{MSE}} $的值, 其中$ {\cal{L}}_{\rm{r}} $表示NPDE的残差的平均值, $ {\cal{L}}_{\rm{u}} $表示NPDE在初始值和边界上的误差的平均值

Fig. 5. When the PINNs model is applied, the value of the function $ {\rm{MSE}} $ is lost throughout the iteration, where $ {\cal{L}}_{\rm{r}} $ represents the average value of the residual error of NPDE, and $ {\cal{L}}_{\rm{u}} $ represents the average value of the error of NPDE at the initial value and the boundary

下载: 全尺寸图片幻灯片

图 4 应用PINNs模型时, 每一个隐藏层的梯度分布, 其中绿色、蓝色、黄色曲线分别对应${g_{\rm{f}}}, \;{g_{\rm{I}}}$和$ {g_{\rm{B}}} $

Fig. 4. When PINNs model is applied, the gradient distribution of each hidden layer is ${g_{\rm{f}}},\; {g_{\rm{I}}}$, and $ {g_{\rm{B}}} $ by color (green, blue, yellow)

下载: 全尺寸图片幻灯片

图 6 (a) CH方程的peakon解((8)式); (b)应用GOPINNs模型模拟时的预测解; (c) GOPINNs模型的绝对误差

Fig. 6. (a) Peakon solution of CH equation (Eq. (8)); (b) predicted solution when using GOPINNs model; (c) absolute error of GOPINNs model

下载: 全尺寸图片幻灯片

图 7 应用GOPINNs模型时, 每一个隐藏层的梯度分布, 其中绿色、蓝色、黄色曲线分别对应$ {g_{\rm{f}}}, {g_{\rm{I}}}, {g_{\rm{B}}} $

Fig. 7. When GOPINNs model is applied, the gradient distribution of each hidden layer is $ {g_{\rm{f}}}, {g_{\rm{I}}}, {g_{\rm{B}}} $ by color (green, blue, yellow)

下载: 全尺寸图片幻灯片

图 8 应用GOPINNs模型时, 整个迭代过程中损失函数${\rm{MSE}}$的值, 其中$ {\cal{L}}_{\rm{r}} $表示NPDE的残差的平均值, $ {\cal{L}}_{\rm{u}} $表示NPDE在初始值和边界上的误差的平均值

Fig. 8. When the GOPINNs model is applied, the value of the function $ {\rm{MSE}} $ is lost during the whole iteration, where $ {\cal{L}}_{\rm{r}} $ represents the average value of residual error of NPDE, and $ {\cal{L}}_{\rm{u}} $ represents the average value of the error of NPDE at the initial value and the boundary

下载: 全尺寸图片幻灯片

图 9 (a)导数非线性薛定谔方程的有理波解((11)式); (b) 使用GOPINNs模型模拟的预测解; (c) GOPINNs模型的绝对误差

Fig. 9. (a) Rational wave solution of the derivative nonlinear Schrödinger equation (Eq. (11)); (b) predicted solution simulated by the GOPINNs model; (c) absolute error of the GOPINNs model

下载: 全尺寸图片幻灯片

图 10 (a)导数非线性薛定谔方程的一阶怪波解((12)式); (b)应用GOPINNs模型模拟的预测解; (c) GOPINNs模型的绝对误差

Fig. 10. (a) First order strange wave solution of derivative nonlinear Schrödinger equation (Eq. (12)); (b) predicted solution simulated by GOPINNs model; (c) absolute error of GOPINNs model

下载: 全尺寸图片幻灯片

表 1 两种模型对导数非线性薛定谔方程的有理波解((11)式)的数值预测结果

Table 1. Numerical prediction results of rational wave solution of derivative nonlinear Schrödinger equation (Eq. (11)) by two models

神经网络信息	PINNs模型的相对 ${L_2}$误差	GOPINNs 模型的相对 ${L_2}$误差	两模型模拟求解的时间消耗/s
6层隐藏层\| 50个神经元	0.237	$4.56 \times 10^{-2}$	24564\|15674.4

下载: 导出CSV

表 2 两种模型对导数非线性薛定谔方程的一阶怪波解((12式))的数值预测结果

Table 2. Numerical prediction results of the first order odd wave solution of derivative nonlinear Schrödinger equation (Eq. (12)) by two models

神经网络信息	PINNs模型的相对 ${L_2}$误差	GOPINNs 模型的相对 ${L_2}$误差	两模型模拟方程求解的时间消耗/s
6层隐藏层\| 50个神经元	$0.465$	$8.16 \times 10^{ -2}$	36223.2\|24345.7

下载: 导出CSV

[1]	Linβ T 2001 Comput. Math. Math. Phys. 41 898 Google Scholar
[2]	Vulanović R 1988 Z. Angew. Math. Mech. 5 428 Google Scholar
[3]	Vulanović R, Nhan T A 2020 J. Comput. Appl. Math. 386 125495 Google Scholar
[4]	Gowrisankar S, Srinivasan N 2019 Appl. Math. Comput. 346 385 Google Scholar
[5]	Nie F, Wang H, Song Q, Zhao Y, Shen J, Gong M 2022 Int. J. Multiphase Flow 152 104067 Google Scholar
[6]	Lagendijk L R, Biemond J, et al. 1988 International Conference on Acoustics New York, USA, April 11–14, 1988 p60916260
[7]	Simon H 1980 Cognitive Science 4 33 Google Scholar
[8]	Busemeyer J 2015 Cognition 135 43 Google Scholar
[9]	Sharma N, Jain V, Mishra A 2018 Procedia Comput. Sci. 132 377 Google Scholar
[10]	Gu J X, Wang Z H, Jason K, Ma L y, Amir S, Shuai B, et al. 2018 Pattern Recognit. 77 354 Google Scholar
[11]	He K, Zhang X, Ren S, Sun J 2016 Las Vegas Proceedings of the IEEE Conference on Computer Vision and Pattern RecognitionLas Vegas, USA, June 27–30, 2016 p770
[12]	Dieleman S, Zen H, Simonyan K, Vinyals O, Graves A, et al 2016 arXiv: 1609.03499 [cs.SD]
[13]	Heaton J, Goodfellow I, Bengio Y, Courville A 2018 Genet Program Evolvable Mach. 19 305 Google Scholar
[14]	Alipanahi B, Delong A, Weirauch T M, Frey J B 2015 Nat. Biotechnol. 33 831 Google Scholar
[15]	Han J, Jentzen A, Weinan E 2018 Proc. Natl. Acad. Sci. 115 8505 Google Scholar
[16]	Rudy H S, Brunton L S, Proctor L J, Kutz N 2017 Sci. Adv. 3 e1602614 Google Scholar
[17]	Raissi M, Karniadakis G E 2018 J. Comput. Phys. 357 125 Google Scholar
[18]	Weinan E, Han J Q, Jentzen A 2017 Commun. Math. Stat. 5 349 Google Scholar
[19]	Sirignano J, Spiliopoulos K 2018 J. Comput. Phys. 375 1339 Google Scholar
[20]	Raissi M, Perdikaris P, Karniadakis G E 2019 J. Comput. Phys. 378 686 Google Scholar
[21]	Jagtap A D, Kharazmi E, Karniadakis G E 2020 Proc. R. Soc. A 476 20200334 Google Scholar
[22]	Revanth M, Susanta G 2021 arXiv: 2106 07606 [math.NA]
[23]	Li J, Chen Y 2020 Commun.Theor. Phys. 72 105005 Google Scholar
[24]	Li J, Chen Y 2020 Commun. Theor. Phys. 72 115003 Google Scholar
[25]	Li J, Chen Y 2021 Commun. Theor. Phys. 73 015001 Google Scholar
[26]	Pu J C, Li J, Chen Y 2021 Chin. Phys. B 30 060202 Google Scholar
[27]	Pu J C, Li J, Chen Y 2021 Nonlinear Dyn. 105 1723 Google Scholar
[28]	Pu J C, Chen Y 2022 Chaos, Solitons Fractals 160 112182 Google Scholar
[29]	Lin S N, Chen Y 2022 J. Comput. Phys. 41 898 Google Scholar
[30]	Ling L M, Mo Y F, Zeng D L 2022 Phys. Lett. A 421 127739 Google Scholar
[31]	He J S, Wang J L 2022 Phys. Lett. A 452 128432 Google Scholar
[32]	Wang L, Yan Z Y 2021 Phys. Lett. A 404 127408 Google Scholar
[33]	Wang L, Yan Z Y 2022 Phys. Lett. A 450 128373 Google Scholar
[34]	Fang Y, Wu G Z, Wang Y Y, et al. 2021 Nonlinear Dyn 105 603 Google Scholar
[35]	Zhou Z J, Yan Z Y 2021 Phys. Lett. A 387 127010 Google Scholar
[36]	Wang L, Yan Z Y 2021 Physica D 428 133037 Google Scholar
[37]	Bai Y, Chaolu T, Bilige S 2021 Nonlinear Dyn. 105 3439 Google Scholar
[38]	Wu G Z, Fang Y, Dai C Q, et al. 2021 Chaos, Solitons Fractals 152 111393 Google Scholar
[39]	Li J H, Li B 2021 Commun. Theor. Phys. 73 125001 Google Scholar
[40]	Li J H, Chen J C, Li B 2022 Nonlinear Dyn. 107 781 Google Scholar
[41]	Li J H, Li B 2022 Chaos, Solitons Fractals 164 112712 Google Scholar
[42]	Fang Y, Wu G Z, Dai C Q, et al. 2022 Chaos, Solitons Fractals 158 112118 Google Scholar
[43]	Wu G Z, Fang Y, Dai C Q, et al. 2022 Chaos, Solitons Fractals 159 112143 Google Scholar
[44]	Yuan L, Ni Y Q, Deng X Y, Hao S 2022 J. Comput. Phys. 462 111260 Google Scholar
[45]	Zeng S J, Zhang Z, Zou Q S 2022 J. Comput. Phys. 463 111232 Google Scholar
[46]	Samadi-koucheksaraee A, Ahmadianfar I, Bozorg-Haddad O, et al. 2019 Water Resour. Manage. 33 603 Google Scholar
[47]	Marcucci G, Pierangeli D, Conti C 2020 Phys. Rev. Lett. 125 093901 Google Scholar
[48]	Kingma D P, Jimmy B 2014 arXiv: 1412 6980 [cs.LG]
[49]	Glorot X, Bengio Y 2010 Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics Chia Laguna Resort, Sardinia, Italy, March 31, 2010 pp249–256
[50]	Camassa R, Holm D 1993 Phys. Rev. Lett. 71 1661 Google Scholar
[51]	Metin G, Atalay K 1998 J. Math. Phys. 39 2103 Google Scholar
[52]	Takayuki T, Miki W 1999 Phys. Lett. A 257 53 Google Scholar
[53]	Xu S W, He J S, Wang L H 2011 J. Phys. A: Math. Theor. 44 305203 Google Scholar

计量

文章访问数: 2658
PDF下载量: 131
被引次数: 0

姓名
邮箱
手机号码
标题
留言内容
验证码

神经网络信息	PINNs模型的相对 ${L_2}$误差	GOPINNs 模型的相对 ${L_2}$误差	两模型模拟求解的时间消耗/s
6层隐藏层\| 50个神经元	0.237	$4.56 \times 10^{-2}$	24564\|15674.4

神经网络信息	PINNs模型的相对 ${L_2}$误差	GOPINNs 模型的相对 ${L_2}$误差	两模型模拟方程求解的时间消耗/s
6层隐藏层\| 50个神经元	$0.465$	$8.16 \times 10^{ -2}$	36223.2\|24345.7

搜索

留言板