-
提出了一种基于物理驱动的融合注意力机制的新型卷积网络单像素成像方法. 通过将结合通道与空间注意力机制的模块集成到一个随机初始化的卷积网络中, 利用单像素成像的物理模型约束网络, 实现了高质量的图像重建. 具体来说, 将空间与通道两个维度的注意力机制集成为一个模块, 引入到多尺度U-net卷积网络的各层中, 通过这种方式, 不仅可以利用注意力机制在三维数据立方中提供的关键权重信息, 还充分结合了U-net网络在不同空间频率下强大的特征提取能力. 这一创新方法能够有效捕捉图像细节, 抑制背景噪声, 提升图像重建质量. 实验结果表明, 针对低采样率条件下的图像重建, 与传统非预训练网络相比, 融合注意力机制的方案不仅在直观上图像细节重建得更好, 而且在定量的评价指标(如峰值信噪比和结构相似性)上均表现出显著优势, 验证了其在单像素成像中的有效性与应用前景.This paper presents a novel convolutional neural network-based single-pixel imaging method that integrates a physics-driven fusion attention mechanism. By incorporating a module that combines both channel attention mechanism and spatial attention mechanism into a randomly initialized convolutional network, the method utilizes the physical model constraints of single-pixel imaging to achieve high-quality image reconstruction. Specifically, the spatial and channel attention mechanism are combined into a single module and introduced into various layers of a multi-scale U-net convolutional network. In the spatial attention mechanism, we extract the attention weight features of each spatial region of the pooled feature map by using convolution. In the channel attention mechanism, we pool the three-dimensional feature map into a single-channel signal and input it into a two-layer fully connected network to obtain the attention weight information for each channel. This approach not only uses the critical weighting information provided by the attention mechanism in the three-dimensional data cube but also fully integrates the powerful feature extraction capabilities of the U-net network across different spatial frequencies. This innovative method can effectively capture image details, suppress background noise, and improve image reconstruction quality. During the experimental phase, we employ the optical path of single-pixel imaging to acquire bucket signals for two target images, "snowflake" and "basket". By inputting any noisy image into a randomly initialized neural network with attention mechanism, and using the mean square error between simulated bucket signal and actual bucket signal, we physically constrain the convergence of the network. Ultimately, we achieve a reconstructed image that adheres to the physical model. The experimental results demonstrate that under low sampling rate conditions, the scheme of integrating the attention mechanism can not only intuitively reconstruct image details better, but also demonstrate significant advantages in quantitative evaluation metrics such as peak signal-to-noise ratio (PSNR) and structural similarity (SSIM), confirming its effectiveness and potential application in single-pixel imaging.
-
Keywords:
- single-pixel imaging /
- attention mechanisms /
- convolutional neural networks /
- image reconstruction
[1] Kilcullen P, Ozaki T, Liang J 2022 Nat. Commun. 13 7879
Google Scholar
[2] Hahamovich E, Monin S, Hazan Y, Rosenthal A 2021 Nat. Commun. 12 4516
Google Scholar
[3] Shapiro J H 2008 Phys. Rev. A 78 061802
Google Scholar
[4] Ferri F, Magatti D, Gatti A, Bache M, Brambilla E, Lugiato L 2005 Phys. Rev. Lett. 94 183602
Google Scholar
[5] Wang F, Wang C, Deng C, Han S, Situ G 2022 Photon. Res. 10 104
Google Scholar
[6] Pan L, Shen Y, Qi J, Shi J, Feng X 2023 Opt. Express 31 13943
Google Scholar
[7] Song K, Bian Y, Wang D, Li R, Wu K, Liu H, Qin C, Hu J, Xiao L 2024 Laser & Photonics Rev. published online 2401397
[8] Zhao X S, Yu C, Wang C, Li T, Liu B, Lu H, Zhang R, Dou X, Zhang J, Pan J W 2024 Appl. Phys. Lett. 125 211103
Google Scholar
[9] Karpowicz N, Zhong H, Xu J, Lin K I, Hwang J S, Zhang X C 2005 Semicond. Sci. Tech. 20 S293
Google Scholar
[10] Simões M, Vaz P, Cortez A F V 2024. arXiv: 2411.03907 [physics.ins-det]
[11] Shwartz S 2021 Sci. Bull. 66 857
Google Scholar
[12] Olbinado M P, Paganin D M, Cheng Y, Rack A 2021 Optica 8 1538
Google Scholar
[13] Clemente P, Durán V, Tajahuerce E, Andrés P, Climent V, Lancis J 2013 Opt. Lett. 38 2524
Google Scholar
[14] Jiang W, Yin Y, Jiao J, Zhao X, Sun B 2022 Photon. Res. 10 2157
Google Scholar
[15] Gibson G M, Sun B, Edgar M P, Phillips D B, Hempler N, Maker G T, Malcolm G P A, Padgett M J 2017 Opt. Express 25 2998
Google Scholar
[16] Zhou L, Xiao Y, Chen W 2023 Opt. Express 31 23027
Google Scholar
[17] Xu Y, Lu L, Saragadam V, Kelly K F 2024 Nat. Commun. 15 1456
Google Scholar
[18] Li J, Li X, Yardimci N T, Hu J, Li Y, Chen J, Hung Y C, Jarrahi M, Ozcan A 2023 Nat. Commun. 14 6791
Google Scholar
[19] Li S, Liu X, Xiao Y, Ma Y, Yang J, Zhu K, Tian X 2023 Opt. Express 31 4712
Google Scholar
[20] Zheng P, Dai Q, Li Z, Ye Z, Xiong J, Liu H C, Zheng G, Zhang S 2021 Sci. Adv. 7 eabg0363
Google Scholar
[21] Katz O, Bromberg Y, Silberberg Y 2009 Appl. Phys. Lett. 95 131110
Google Scholar
[22] López-García L, Cruz-Santos W, GarcíaArellano A, Filio-Aguilar P, Cisneros-Martínez J A, Ramos-García R 2022 Opt. Express 30 13714
Google Scholar
[23] Zhang Z, Ma X, Zhong J 2015 Nat. Commun. 6 6225
Google Scholar
[24] Donoho D 2006 IEEE Trans. Inf. Theory 52 1289
Google Scholar
[25] Duarte M F, Davenport M A, Takhar D, Laska J N, Sun T, Kelly K F, Baraniuk R G 2008 IEEE Signal Process Mag. 25 83
Google Scholar
[26] Huang L, Luo R, Liu X, Hao X 2022 Light Sci. Appl. 11 61
Google Scholar
[27] Figueiredo M A T, Nowak R D, Wright S J 2007 IEEE J. Sel. Top. Signal Process. 11 586
[28] Pioneers A 2024 Nat. Mach. Intell. 6 1271
Google Scholar
[29] 查文舒, 李道伦, 沈路航, 张雯, 刘旭亮 2022 力学学报 54 543
Google Scholar
Zha W S, Li D L, Shen L H, Zhang W, Liu X L 2022 Chinese Journal of Theoretical and Applied Mechanics 54 543
Google Scholar
[30] Zhang H, Wang J, Zhang Y, Du X, Wu H, Zhang T 2024 Astronomical Techniques and Instruments 1 1
[31] van Leeuwen C, Podareanu D, Codreanu V, Cai M X, Berg A, Zwart S P, Stoffer R, Veerman M, van Heerwaarden C, Otten S, Caron S, Geng C, Ambrosetti F, Bonvin A M J J 2020 arXiv: 2004.03454[cs.CE]
[32] Barbastathis G, Ozcan A, Situ G 2019 Optica 6 921
Google Scholar
[33] Ruget A, Moodley C, Forbes A, Leach J 2024 Opt. Express 32 41057
Google Scholar
[34] Wetzstein G, Ozcan A, Gigan S, Fan S, Englund D, Soljačić M, Denz C, Miller D A B, Psaltis D 2020 Nature 588 39
Google Scholar
[35] Lyu M, Wang W, Wang H, Wang H, Li G, Chen N, Situ G 2017 Sci. Rep. 7 17865
Google Scholar
[36] Zhang X, Deng C, Wang C, Wang F, Situ G 2023 ACS Photonics 10 2363
Google Scholar
[37] Li J, Li Y, Li J, Zhang Q, Li J 2020 Opt. Express 28 22992
Google Scholar
[38] Wang F, Wang C, Chen M, Gong W, Zhang Y, Han S, Situ G 2022 Light Sci. Appl. 11 1
Google Scholar
[39] Peng L, Xie S, Qin T, Cao L, Bian L 2023 Opt. Lett. 48 2527
Google Scholar
[40] Liu H, Bian L, Zhang J 2023 Opt. Laser Technol. 157 108600
Google Scholar
[41] Liu X, Han T, Zhou C, Huang J, Ju M, Xu B, Song L 2023 Opt. Express 31 9945
Google Scholar
[42] Hammernik K, Küstner T, Yaman B, Huang Z, Rueckert D, Knoll F, Akçakaya M 2023 IEEE Signal Process Mag. 40 98
[43] Wang P, Chen P, Yuan Y, Liu D, Huang Z, Hou X, Cottrell G W 2017. arXiv: 1702.08502[cs.CV]
[44] Ulyanov D, Vedaldi A, Lempitsky V 2020 IJCV 128 1867
Google Scholar
[45] Ren W, Nie X, Peng T, Scully M O 2022 Opt. Express 30 47921
Google Scholar
[46] Zhang H, Sindagi V, Patel V M 2020 IEEE Trans. Circuits Syst. Video Technol. 30 3943
Google Scholar
[47] Lv W, Xiong J, Shi J, Huang Y, Qin S 2021 J. Intell. Manuf. 32 441
Google Scholar
[48] Zhang H, Wang Z, Liu D 2014 IEEE Transactions on Neural Networks and Learning Systems 25 1229
Google Scholar
[49] Baozhou Z, Hofstee P, Lee J, Al-Ars Z 2021 arXiv: 2108.08205 [cs.CV]
[50] Karim N, Rahnavard N 2021 arXiv: 2107.01330[cs.CV]
[51] Hoshi I, Shimobaba T, Kakue T, Ito T 2020 Opt. Express 28 34069
Google Scholar
[52] Stollenga M, Masci J, Gomez F, Schmidhuber J 2014 arXiv: 1407.3068[cs.CV]
[53] Zhang Y, Li K, Li K, Wang L, Zhong B, Fu Y 2018 arXiv: 1807.02758[cs.CV]
[54] Liao X, He L, Mao J, Xu M 2024 Remote Sensing 16 1688
Google Scholar
[55] Yu W K, Wang S F, Shang K Q 2024 Sensors 24 1012
Google Scholar
[56] Ronneberger O, Fischer P, Brox T 2015 arXiv: 1505.04597[cs.CV]
[57] Meng Z, Yu Z, Xu K, Yuan X 2021 arXiv: 2108.12654 [eess.IV]
[58] Ferri F, Magatti D, Lugiato L A, Gatti A 2010 Phys. Rev. Lett. 104 253603
Google Scholar
[59] Lin J, Yan Q, Lu S, Zheng Y, Sun S, Wei Z 2022 Photonics 9 343
Google Scholar
-
图 2 融合注意力机制的U-net卷积神经网络结构示意图 (a) U-net结构的卷积网络; (b) CBAM模块结构总览; (c)空间注意力机制模块; (d)通道注意力机制模块
Fig. 2. Schematic diagram of U-net convolutional neural network structure with integrated attention mechanism: (a) Convolutional neural networks of a U-net architecture; (b) overall structure of CBAM; (c) spatial attention module; (d) channel attention module
图 6 不同迭代次数下PSNR与损失函数的变化对比 (a)两种方案重建图像的PSNR随迭代次数的变化; (b)本文方案的损失函数在不同初始学习率下随迭代次数的变化
Fig. 6. Comparison of PSNR and loss function under different iterations: (a) The PSNR of the reconstructed images of the two schemes varies with iterations; (b) the loss function of our scheme varies with iterations under different initial learning rates.
-
[1] Kilcullen P, Ozaki T, Liang J 2022 Nat. Commun. 13 7879
Google Scholar
[2] Hahamovich E, Monin S, Hazan Y, Rosenthal A 2021 Nat. Commun. 12 4516
Google Scholar
[3] Shapiro J H 2008 Phys. Rev. A 78 061802
Google Scholar
[4] Ferri F, Magatti D, Gatti A, Bache M, Brambilla E, Lugiato L 2005 Phys. Rev. Lett. 94 183602
Google Scholar
[5] Wang F, Wang C, Deng C, Han S, Situ G 2022 Photon. Res. 10 104
Google Scholar
[6] Pan L, Shen Y, Qi J, Shi J, Feng X 2023 Opt. Express 31 13943
Google Scholar
[7] Song K, Bian Y, Wang D, Li R, Wu K, Liu H, Qin C, Hu J, Xiao L 2024 Laser & Photonics Rev. published online 2401397
[8] Zhao X S, Yu C, Wang C, Li T, Liu B, Lu H, Zhang R, Dou X, Zhang J, Pan J W 2024 Appl. Phys. Lett. 125 211103
Google Scholar
[9] Karpowicz N, Zhong H, Xu J, Lin K I, Hwang J S, Zhang X C 2005 Semicond. Sci. Tech. 20 S293
Google Scholar
[10] Simões M, Vaz P, Cortez A F V 2024. arXiv: 2411.03907 [physics.ins-det]
[11] Shwartz S 2021 Sci. Bull. 66 857
Google Scholar
[12] Olbinado M P, Paganin D M, Cheng Y, Rack A 2021 Optica 8 1538
Google Scholar
[13] Clemente P, Durán V, Tajahuerce E, Andrés P, Climent V, Lancis J 2013 Opt. Lett. 38 2524
Google Scholar
[14] Jiang W, Yin Y, Jiao J, Zhao X, Sun B 2022 Photon. Res. 10 2157
Google Scholar
[15] Gibson G M, Sun B, Edgar M P, Phillips D B, Hempler N, Maker G T, Malcolm G P A, Padgett M J 2017 Opt. Express 25 2998
Google Scholar
[16] Zhou L, Xiao Y, Chen W 2023 Opt. Express 31 23027
Google Scholar
[17] Xu Y, Lu L, Saragadam V, Kelly K F 2024 Nat. Commun. 15 1456
Google Scholar
[18] Li J, Li X, Yardimci N T, Hu J, Li Y, Chen J, Hung Y C, Jarrahi M, Ozcan A 2023 Nat. Commun. 14 6791
Google Scholar
[19] Li S, Liu X, Xiao Y, Ma Y, Yang J, Zhu K, Tian X 2023 Opt. Express 31 4712
Google Scholar
[20] Zheng P, Dai Q, Li Z, Ye Z, Xiong J, Liu H C, Zheng G, Zhang S 2021 Sci. Adv. 7 eabg0363
Google Scholar
[21] Katz O, Bromberg Y, Silberberg Y 2009 Appl. Phys. Lett. 95 131110
Google Scholar
[22] López-García L, Cruz-Santos W, GarcíaArellano A, Filio-Aguilar P, Cisneros-Martínez J A, Ramos-García R 2022 Opt. Express 30 13714
Google Scholar
[23] Zhang Z, Ma X, Zhong J 2015 Nat. Commun. 6 6225
Google Scholar
[24] Donoho D 2006 IEEE Trans. Inf. Theory 52 1289
Google Scholar
[25] Duarte M F, Davenport M A, Takhar D, Laska J N, Sun T, Kelly K F, Baraniuk R G 2008 IEEE Signal Process Mag. 25 83
Google Scholar
[26] Huang L, Luo R, Liu X, Hao X 2022 Light Sci. Appl. 11 61
Google Scholar
[27] Figueiredo M A T, Nowak R D, Wright S J 2007 IEEE J. Sel. Top. Signal Process. 11 586
[28] Pioneers A 2024 Nat. Mach. Intell. 6 1271
Google Scholar
[29] 查文舒, 李道伦, 沈路航, 张雯, 刘旭亮 2022 力学学报 54 543
Google Scholar
Zha W S, Li D L, Shen L H, Zhang W, Liu X L 2022 Chinese Journal of Theoretical and Applied Mechanics 54 543
Google Scholar
[30] Zhang H, Wang J, Zhang Y, Du X, Wu H, Zhang T 2024 Astronomical Techniques and Instruments 1 1
[31] van Leeuwen C, Podareanu D, Codreanu V, Cai M X, Berg A, Zwart S P, Stoffer R, Veerman M, van Heerwaarden C, Otten S, Caron S, Geng C, Ambrosetti F, Bonvin A M J J 2020 arXiv: 2004.03454[cs.CE]
[32] Barbastathis G, Ozcan A, Situ G 2019 Optica 6 921
Google Scholar
[33] Ruget A, Moodley C, Forbes A, Leach J 2024 Opt. Express 32 41057
Google Scholar
[34] Wetzstein G, Ozcan A, Gigan S, Fan S, Englund D, Soljačić M, Denz C, Miller D A B, Psaltis D 2020 Nature 588 39
Google Scholar
[35] Lyu M, Wang W, Wang H, Wang H, Li G, Chen N, Situ G 2017 Sci. Rep. 7 17865
Google Scholar
[36] Zhang X, Deng C, Wang C, Wang F, Situ G 2023 ACS Photonics 10 2363
Google Scholar
[37] Li J, Li Y, Li J, Zhang Q, Li J 2020 Opt. Express 28 22992
Google Scholar
[38] Wang F, Wang C, Chen M, Gong W, Zhang Y, Han S, Situ G 2022 Light Sci. Appl. 11 1
Google Scholar
[39] Peng L, Xie S, Qin T, Cao L, Bian L 2023 Opt. Lett. 48 2527
Google Scholar
[40] Liu H, Bian L, Zhang J 2023 Opt. Laser Technol. 157 108600
Google Scholar
[41] Liu X, Han T, Zhou C, Huang J, Ju M, Xu B, Song L 2023 Opt. Express 31 9945
Google Scholar
[42] Hammernik K, Küstner T, Yaman B, Huang Z, Rueckert D, Knoll F, Akçakaya M 2023 IEEE Signal Process Mag. 40 98
[43] Wang P, Chen P, Yuan Y, Liu D, Huang Z, Hou X, Cottrell G W 2017. arXiv: 1702.08502[cs.CV]
[44] Ulyanov D, Vedaldi A, Lempitsky V 2020 IJCV 128 1867
Google Scholar
[45] Ren W, Nie X, Peng T, Scully M O 2022 Opt. Express 30 47921
Google Scholar
[46] Zhang H, Sindagi V, Patel V M 2020 IEEE Trans. Circuits Syst. Video Technol. 30 3943
Google Scholar
[47] Lv W, Xiong J, Shi J, Huang Y, Qin S 2021 J. Intell. Manuf. 32 441
Google Scholar
[48] Zhang H, Wang Z, Liu D 2014 IEEE Transactions on Neural Networks and Learning Systems 25 1229
Google Scholar
[49] Baozhou Z, Hofstee P, Lee J, Al-Ars Z 2021 arXiv: 2108.08205 [cs.CV]
[50] Karim N, Rahnavard N 2021 arXiv: 2107.01330[cs.CV]
[51] Hoshi I, Shimobaba T, Kakue T, Ito T 2020 Opt. Express 28 34069
Google Scholar
[52] Stollenga M, Masci J, Gomez F, Schmidhuber J 2014 arXiv: 1407.3068[cs.CV]
[53] Zhang Y, Li K, Li K, Wang L, Zhong B, Fu Y 2018 arXiv: 1807.02758[cs.CV]
[54] Liao X, He L, Mao J, Xu M 2024 Remote Sensing 16 1688
Google Scholar
[55] Yu W K, Wang S F, Shang K Q 2024 Sensors 24 1012
Google Scholar
[56] Ronneberger O, Fischer P, Brox T 2015 arXiv: 1505.04597[cs.CV]
[57] Meng Z, Yu Z, Xu K, Yuan X 2021 arXiv: 2108.12654 [eess.IV]
[58] Ferri F, Magatti D, Lugiato L A, Gatti A 2010 Phys. Rev. Lett. 104 253603
Google Scholar
[59] Lin J, Yan Q, Lu S, Zheng Y, Sun S, Wei Z 2022 Photonics 9 343
Google Scholar
计量
- 文章访问数: 752
- PDF下载量: 17
- 被引次数: 0