-
In recent years, physics-informed neural networks (PINNs) have attracted more and more attention for their ability to quickly obtain high-precision data-driven solutions with only a small amount of data. However, although this model has good results in some nonlinear problems, it still has some shortcomings. For example, the unbalanced back-propagation gradient calculation results in the intense oscillation of the gradient value during the model training, which is easy to lead to the instability of the prediction accuracy. Based on this, we propose a gradient-optimized physics-informed neural networks (GOPINNs) model in this paper, which proposes a new neural network structure and balances the interaction between different terms in the loss function during model training through gradient statistics, so as to make the new proposed network structure more robust to gradient fluctuations. In this paper, taking Camassa-Holm (CH) equation and DNLS equation as examples, GOPINNs is used to simulate the peakon solution of CH equation, the rational wave solution of DNLS equation and the rogue wave solution of DNLS equation. The numerical results show that the GOPINNs can effectively smooth the gradient of the loss function in the calculation process, and obtain a higher precision solution than the original PINNs. In conclusion, our work provides new insights for optimizing the learning performance of neural networks, and saves more than one third of the time in simulating the complex CH equation and the DNLS equation, and improves the prediction accuracy by nearly ten times.
-
Keywords:
- physics-informed neural networks /
- gradient optimization /
- Camassa-Holm equation /
- derivative nonlinear Schrödinger equation /
- peakon solution /
- rogue wave solution
[1] Linβ T 2001 Comput. Math. Math. Phys. 41 898Google Scholar
[2] Vulanović R 1988 Z. Angew. Math. Mech. 5 428Google Scholar
[3] Vulanović R, Nhan T A 2020 J. Comput. Appl. Math. 386 125495Google Scholar
[4] Gowrisankar S, Srinivasan N 2019 Appl. Math. Comput. 346 385Google Scholar
[5] Nie F, Wang H, Song Q, Zhao Y, Shen J, Gong M 2022 Int. J. Multiphase Flow 152 104067Google Scholar
[6] Lagendijk L R, Biemond J, et al. 1988 International Conference on Acoustics New York, USA, April 11–14, 1988 p60916260
[7] Simon H 1980 Cognitive Science 4 33Google Scholar
[8] Busemeyer J 2015 Cognition 135 43Google Scholar
[9] Sharma N, Jain V, Mishra A 2018 Procedia Comput. Sci. 132 377Google Scholar
[10] Gu J X, Wang Z H, Jason K, Ma L y, Amir S, Shuai B, et al. 2018 Pattern Recognit. 77 354Google Scholar
[11] He K, Zhang X, Ren S, Sun J 2016 Las Vegas Proceedings of the IEEE Conference on Computer Vision and Pattern RecognitionLas Vegas, USA, June 27–30, 2016 p770
[12] Dieleman S, Zen H, Simonyan K, Vinyals O, Graves A, et al 2016 arXiv: 1609.03499 [cs.SD]
[13] Heaton J, Goodfellow I, Bengio Y, Courville A 2018 Genet Program Evolvable Mach. 19 305Google Scholar
[14] Alipanahi B, Delong A, Weirauch T M, Frey J B 2015 Nat. Biotechnol. 33 831Google Scholar
[15] Han J, Jentzen A, Weinan E 2018 Proc. Natl. Acad. Sci. 115 8505Google Scholar
[16] Rudy H S, Brunton L S, Proctor L J, Kutz N 2017 Sci. Adv. 3 e1602614Google Scholar
[17] Raissi M, Karniadakis G E 2018 J. Comput. Phys. 357 125Google Scholar
[18] Weinan E, Han J Q, Jentzen A 2017 Commun. Math. Stat. 5 349Google Scholar
[19] Sirignano J, Spiliopoulos K 2018 J. Comput. Phys. 375 1339Google Scholar
[20] Raissi M, Perdikaris P, Karniadakis G E 2019 J. Comput. Phys. 378 686Google Scholar
[21] Jagtap A D, Kharazmi E, Karniadakis G E 2020 Proc. R. Soc. A 476 20200334Google Scholar
[22] Revanth M, Susanta G 2021 arXiv: 2106 07606 [math.NA]
[23] Li J, Chen Y 2020 Commun.Theor. Phys. 72 105005Google Scholar
[24] Li J, Chen Y 2020 Commun. Theor. Phys. 72 115003Google Scholar
[25] Li J, Chen Y 2021 Commun. Theor. Phys. 73 015001Google Scholar
[26] Pu J C, Li J, Chen Y 2021 Chin. Phys. B 30 060202Google Scholar
[27] Pu J C, Li J, Chen Y 2021 Nonlinear Dyn. 105 1723Google Scholar
[28] Pu J C, Chen Y 2022 Chaos, Solitons Fractals 160 112182Google Scholar
[29] Lin S N, Chen Y 2022 J. Comput. Phys. 41 898Google Scholar
[30] Ling L M, Mo Y F, Zeng D L 2022 Phys. Lett. A 421 127739Google Scholar
[31] He J S, Wang J L 2022 Phys. Lett. A 452 128432Google Scholar
[32] Wang L, Yan Z Y 2021 Phys. Lett. A 404 127408Google Scholar
[33] Wang L, Yan Z Y 2022 Phys. Lett. A 450 128373Google Scholar
[34] Fang Y, Wu G Z, Wang Y Y, et al. 2021 Nonlinear Dyn 105 603Google Scholar
[35] Zhou Z J, Yan Z Y 2021 Phys. Lett. A 387 127010Google Scholar
[36] Wang L, Yan Z Y 2021 Physica D 428 133037Google Scholar
[37] Bai Y, Chaolu T, Bilige S 2021 Nonlinear Dyn. 105 3439Google Scholar
[38] Wu G Z, Fang Y, Dai C Q, et al. 2021 Chaos, Solitons Fractals 152 111393Google Scholar
[39] Li J H, Li B 2021 Commun. Theor. Phys. 73 125001Google Scholar
[40] Li J H, Chen J C, Li B 2022 Nonlinear Dyn. 107 781Google Scholar
[41] Li J H, Li B 2022 Chaos, Solitons Fractals 164 112712Google Scholar
[42] Fang Y, Wu G Z, Dai C Q, et al. 2022 Chaos, Solitons Fractals 158 112118Google Scholar
[43] Wu G Z, Fang Y, Dai C Q, et al. 2022 Chaos, Solitons Fractals 159 112143Google Scholar
[44] Yuan L, Ni Y Q, Deng X Y, Hao S 2022 J. Comput. Phys. 462 111260Google Scholar
[45] Zeng S J, Zhang Z, Zou Q S 2022 J. Comput. Phys. 463 111232Google Scholar
[46] Samadi-koucheksaraee A, Ahmadianfar I, Bozorg-Haddad O, et al. 2019 Water Resour. Manage. 33 603Google Scholar
[47] Marcucci G, Pierangeli D, Conti C 2020 Phys. Rev. Lett. 125 093901Google Scholar
[48] Kingma D P, Jimmy B 2014 arXiv: 1412 6980 [cs.LG]
[49] Glorot X, Bengio Y 2010 Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics Chia Laguna Resort, Sardinia, Italy, March 31, 2010 pp249–256
[50] Camassa R, Holm D 1993 Phys. Rev. Lett. 71 1661Google Scholar
[51] Metin G, Atalay K 1998 J. Math. Phys. 39 2103Google Scholar
[52] Takayuki T, Miki W 1999 Phys. Lett. A 257 53Google Scholar
[53] Xu S W, He J S, Wang L H 2011 J. Phys. A: Math. Theor. 44 305203Google Scholar
-
图 5 应用PINNs模型时, 整个迭代过程中损失函数
$ {\rm{MSE}} $ 的值, 其中$ {\cal{L}}_{\rm{r}} $ 表示NPDE的残差的平均值,$ {\cal{L}}_{\rm{u}} $ 表示NPDE在初始值和边界上的误差的平均值Figure 5. When the PINNs model is applied, the value of the function
$ {\rm{MSE}} $ is lost throughout the iteration, where$ {\cal{L}}_{\rm{r}} $ represents the average value of the residual error of NPDE, and$ {\cal{L}}_{\rm{u}} $ represents the average value of the error of NPDE at the initial value and the boundary图 8 应用GOPINNs模型时, 整个迭代过程中损失函数
${\rm{MSE}}$ 的值, 其中$ {\cal{L}}_{\rm{r}} $ 表示NPDE的残差的平均值,$ {\cal{L}}_{\rm{u}} $ 表示NPDE在初始值和边界上的误差的平均值Figure 8. When the GOPINNs model is applied, the value of the function
$ {\rm{MSE}} $ is lost during the whole iteration, where$ {\cal{L}}_{\rm{r}} $ represents the average value of residual error of NPDE, and$ {\cal{L}}_{\rm{u}} $ represents the average value of the error of NPDE at the initial value and the boundary表 1 两种模型对导数非线性薛定谔方程的有理波解((11)式)的数值预测结果
Table 1. Numerical prediction results of rational wave solution of derivative nonlinear Schrödinger equation (Eq. (11)) by two models
神经网络
信息PINNs模型
的相对
${L_2}$误差GOPINNs
模型的相对
${L_2}$误差两模型模拟
求解的时间
消耗/s6层隐藏层|
50个神经元0.237 $4.56 \times 10^{-2}$ 24564|15674.4 表 2 两种模型对导数非线性薛定谔方程的一阶怪波解((12式))的数值预测结果
Table 2. Numerical prediction results of the first order odd wave solution of derivative nonlinear Schrödinger equation (Eq. (12)) by two models
神经网络
信息PINNs模型
的相对
${L_2}$误差GOPINNs
模型的相对
${L_2}$误差两模型模拟
方程求解的
时间消耗/s6层隐藏层|
50个神经元$0.465$ $8.16 \times 10^{ -2}$ 36223.2|24345.7 -
[1] Linβ T 2001 Comput. Math. Math. Phys. 41 898Google Scholar
[2] Vulanović R 1988 Z. Angew. Math. Mech. 5 428Google Scholar
[3] Vulanović R, Nhan T A 2020 J. Comput. Appl. Math. 386 125495Google Scholar
[4] Gowrisankar S, Srinivasan N 2019 Appl. Math. Comput. 346 385Google Scholar
[5] Nie F, Wang H, Song Q, Zhao Y, Shen J, Gong M 2022 Int. J. Multiphase Flow 152 104067Google Scholar
[6] Lagendijk L R, Biemond J, et al. 1988 International Conference on Acoustics New York, USA, April 11–14, 1988 p60916260
[7] Simon H 1980 Cognitive Science 4 33Google Scholar
[8] Busemeyer J 2015 Cognition 135 43Google Scholar
[9] Sharma N, Jain V, Mishra A 2018 Procedia Comput. Sci. 132 377Google Scholar
[10] Gu J X, Wang Z H, Jason K, Ma L y, Amir S, Shuai B, et al. 2018 Pattern Recognit. 77 354Google Scholar
[11] He K, Zhang X, Ren S, Sun J 2016 Las Vegas Proceedings of the IEEE Conference on Computer Vision and Pattern RecognitionLas Vegas, USA, June 27–30, 2016 p770
[12] Dieleman S, Zen H, Simonyan K, Vinyals O, Graves A, et al 2016 arXiv: 1609.03499 [cs.SD]
[13] Heaton J, Goodfellow I, Bengio Y, Courville A 2018 Genet Program Evolvable Mach. 19 305Google Scholar
[14] Alipanahi B, Delong A, Weirauch T M, Frey J B 2015 Nat. Biotechnol. 33 831Google Scholar
[15] Han J, Jentzen A, Weinan E 2018 Proc. Natl. Acad. Sci. 115 8505Google Scholar
[16] Rudy H S, Brunton L S, Proctor L J, Kutz N 2017 Sci. Adv. 3 e1602614Google Scholar
[17] Raissi M, Karniadakis G E 2018 J. Comput. Phys. 357 125Google Scholar
[18] Weinan E, Han J Q, Jentzen A 2017 Commun. Math. Stat. 5 349Google Scholar
[19] Sirignano J, Spiliopoulos K 2018 J. Comput. Phys. 375 1339Google Scholar
[20] Raissi M, Perdikaris P, Karniadakis G E 2019 J. Comput. Phys. 378 686Google Scholar
[21] Jagtap A D, Kharazmi E, Karniadakis G E 2020 Proc. R. Soc. A 476 20200334Google Scholar
[22] Revanth M, Susanta G 2021 arXiv: 2106 07606 [math.NA]
[23] Li J, Chen Y 2020 Commun.Theor. Phys. 72 105005Google Scholar
[24] Li J, Chen Y 2020 Commun. Theor. Phys. 72 115003Google Scholar
[25] Li J, Chen Y 2021 Commun. Theor. Phys. 73 015001Google Scholar
[26] Pu J C, Li J, Chen Y 2021 Chin. Phys. B 30 060202Google Scholar
[27] Pu J C, Li J, Chen Y 2021 Nonlinear Dyn. 105 1723Google Scholar
[28] Pu J C, Chen Y 2022 Chaos, Solitons Fractals 160 112182Google Scholar
[29] Lin S N, Chen Y 2022 J. Comput. Phys. 41 898Google Scholar
[30] Ling L M, Mo Y F, Zeng D L 2022 Phys. Lett. A 421 127739Google Scholar
[31] He J S, Wang J L 2022 Phys. Lett. A 452 128432Google Scholar
[32] Wang L, Yan Z Y 2021 Phys. Lett. A 404 127408Google Scholar
[33] Wang L, Yan Z Y 2022 Phys. Lett. A 450 128373Google Scholar
[34] Fang Y, Wu G Z, Wang Y Y, et al. 2021 Nonlinear Dyn 105 603Google Scholar
[35] Zhou Z J, Yan Z Y 2021 Phys. Lett. A 387 127010Google Scholar
[36] Wang L, Yan Z Y 2021 Physica D 428 133037Google Scholar
[37] Bai Y, Chaolu T, Bilige S 2021 Nonlinear Dyn. 105 3439Google Scholar
[38] Wu G Z, Fang Y, Dai C Q, et al. 2021 Chaos, Solitons Fractals 152 111393Google Scholar
[39] Li J H, Li B 2021 Commun. Theor. Phys. 73 125001Google Scholar
[40] Li J H, Chen J C, Li B 2022 Nonlinear Dyn. 107 781Google Scholar
[41] Li J H, Li B 2022 Chaos, Solitons Fractals 164 112712Google Scholar
[42] Fang Y, Wu G Z, Dai C Q, et al. 2022 Chaos, Solitons Fractals 158 112118Google Scholar
[43] Wu G Z, Fang Y, Dai C Q, et al. 2022 Chaos, Solitons Fractals 159 112143Google Scholar
[44] Yuan L, Ni Y Q, Deng X Y, Hao S 2022 J. Comput. Phys. 462 111260Google Scholar
[45] Zeng S J, Zhang Z, Zou Q S 2022 J. Comput. Phys. 463 111232Google Scholar
[46] Samadi-koucheksaraee A, Ahmadianfar I, Bozorg-Haddad O, et al. 2019 Water Resour. Manage. 33 603Google Scholar
[47] Marcucci G, Pierangeli D, Conti C 2020 Phys. Rev. Lett. 125 093901Google Scholar
[48] Kingma D P, Jimmy B 2014 arXiv: 1412 6980 [cs.LG]
[49] Glorot X, Bengio Y 2010 Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics Chia Laguna Resort, Sardinia, Italy, March 31, 2010 pp249–256
[50] Camassa R, Holm D 1993 Phys. Rev. Lett. 71 1661Google Scholar
[51] Metin G, Atalay K 1998 J. Math. Phys. 39 2103Google Scholar
[52] Takayuki T, Miki W 1999 Phys. Lett. A 257 53Google Scholar
[53] Xu S W, He J S, Wang L H 2011 J. Phys. A: Math. Theor. 44 305203Google Scholar
Catalog
Metrics
- Abstract views: 4540
- PDF Downloads: 171
- Cited By: 0