-
随着同步辐射技术的发展和光源相干性的提升, 叠层相干衍射成像(ptychography)得到快速发展. 叠层相干衍射成像算法解决了传统相干衍射成像算法收敛速度较慢、容易陷入局部最优解和算法停滞等问题, 具有成像视场大、算法鲁棒性强、对误差容忍性高、应用范围广等优点, 正成为相干衍射成像领域的热点研究方向. 本文首先介绍了叠层相干衍射成像算法提出的背景; 然后详细总结了叠层相干衍射成像算法的发展脉络、主要的算法流程以及应用场景, 并且介绍了叠层相干衍射成像与人工智能结合的新算法及应用潜力; 最后介绍了叠层相干衍射成像算法具体的并行化实现及常用软件包. 本文有助于建立叠层相干衍射成像领域算法本身、人工智能以及计算方法全局研究视角, 对于促进叠层相干衍射成像方法学的系统发展具有重要的参考意义.With the development of synchrotron radiation technology and the improvement of light source coherence, ptychography has developed rapidly. Ptychography algorithm solves the problems of slow convergence and easily falls into the local optimal solution and stagnation of the traditional coherent diffraction imaging algorithm. It has the advantages of large imaging field of view, robustness of algorithm, high tolerance to error and wide range of applications, and is becoming a hot research direction in the field of coherent diffraction imaging. Ptychography reconstructs the complex amplitude distribution and illumination light of the sample by iterative algorithms, which can theoretically reach the resolution of the diffraction limit. It has excellent applications in the fields of wavefront detection, phase imaging and optical metrology. This paper first introduces the background of the proposed ptychography algorithm and briefly describes the problem of coherent diffraction imaging algorithm and its development, and then summarizes the development of ptychography algorithm in detail, mainly including the mainstream algorithm of ptychography and its kernel. This paper then describes in detail the improvement of algorithms corresponding to the improvement of the efficiency of ptychography experiments, correction of position errors and the effect of illumination light multi-modal, and elaborates the algorithm flow. After analyzing the possible intersection of diffraction imaging and neural networks in the field of artificial intelligence, this paper introduces new algorithms with combining ptychography with artificial intelligence. New algorithms with combining ptychography with neural networks will have new potential applications in generality, accuracy and robustness. Finally, a specific parallelization implementation of the ptychography algorithm and common software packages are presented. The logic for writing the parallelization of the algorithm implementation of each package and the corresponding advantages and disadvantages of the packages are described in detail. The characteristics and performance of each package are then listed for reference. This paper helps to establish a global perspective of the algorithm itself, artificial intelligence and computational methods in the field of ptychography, and presents an important reference for systematically developing the ptychography method.
-
Keywords:
- ptychography /
- algorithm /
- imaging methodology /
- artificial intelligence
[1] Bates R 1982 Opt. Stuttg. 61 5
[2] 赵江涛 2020 博士学位论文 (合肥: 中国科学技术大学)
Zhao J T 2020 Ph. D. Dissertation(Hefei: University of Science and Technology of China) (in Chinese)
[3] Bates R, Fright W R 1983 J. Opt. Soc. Am. 73 358
[4] Miao J, Charalambous P, Kirz J, Sayre D 1999 Nature 400 342
Google Scholar
[5] Gerchberg R W, Saxton W 1971 Optik 35 237
[6] Fienup J R 1978 Opt. Lett. 3 27
Google Scholar
[7] Fienup J R 1982 Appl. Opt. 21 2758
Google Scholar
[8] Fienup J R, Wackerman C C 1986 JOSA A 3 1897
Google Scholar
[9] Rodenburg J M, Faulkner H M L 2004 Appl. Phys. Lett. 85 4795
Google Scholar
[10] Faulkner H M L, Rodenburg J M 2004 Phys. Rev. Lett. 93 023903
Google Scholar
[11] Saxton W O 2013 Computer Techniques for Image Processing in Electron Microscopy (Academic Press) pp78–96
[12] 潘兴臣, 刘诚, 陶华, 刘海岗, 朱健强 2020 光学学报 40 0111010
Google Scholar
Pan X C, Liu C, Tao H, Liu H G, Zhu J Q 2020 Acta Opt. Sin. 40 0111010
Google Scholar
[13] Rodenburg J M, Hurst A C, Cullis A G 2007 Ultramicroscopy 107 227
Google Scholar
[14] Rodenburg J M 2008 Adv. Imaging Electro. Phys. 2008 87
Google Scholar
[15] Moxham T E, Laundy D, Dhamgaye V, Fox O J, Sawhney K, Korsunsky A M 2021 Appl. Phys. Lett. 118 104104
Google Scholar
[16] Shemilt L, Verbanis E, Schwenke J, Estandarte A K, Xiong G, Harder R, Parmar N, Yusuf M, Zhang F, Robinson I K 2015 Biophys. J. 108 706
Google Scholar
[17] Bhartiya A, Batey D, Cipiccia S, Shi X, Rau C, Botchway S, Yusuf M, Robinson I K 2021 Chromosome Res. 29 107
Google Scholar
[18] Beckers M, Senkbeil T, Gorniak T, Reese M, Giewekemeyer K, Gleber S C, Salditt T, Rosenhahn A 2011 Phys. Rev. Lett. 107 208101
Google Scholar
[19] D’alfonso A J, Morgan A J, Yan A W C, Wang P, Sawada H, Kirkland A I, Allen L J 2014 Phys. Rev. B 89 064101
Google Scholar
[20] Kane D J 2019 IEEE J. Sel. Top. Quantum Electron. 25 1
Google Scholar
[21] Thibault P, Dierolf M, Bunk O, Menzel A, Pfeiffer F 2009 Ultramicroscopy 109 338
Google Scholar
[22] Maiden A M, Rodenburg J M 2009 Ultramicroscopy 109 1256
Google Scholar
[23] Bunk O, Dierolf M, Kynde S, Johnson I, Marti O, Pfeiffer F 2008 Ultramicroscopy 108 481
Google Scholar
[24] [25] Dierolf M, Thibault P, Menzel A, Kewish C M, Jefimovs K, Schlichting I, König K von, Bunk O, Pfeiffer F 2010 New J. Phys. 12 035017
Google Scholar
[26] Clark J N, Huang X, Harder R J, Robinson I K 2014 Opt. Lett. 39 6066
Google Scholar
[27] Pan X, Liu C, Zhu J 2013 Appl. Phys. Lett. 103 2758
Google Scholar
[28] Sidorenko P, Cohen O 2016 Optica 3 9
Google Scholar
[29] Chen B K, Sidorenko P, Lahav O, Peleg O, Cohen O 2018 Opt. Lett. 43 5379
Google Scholar
[30] Xu W, Xu H, Luo Y, Li T, Shi Y 2016 Opt. Express 24 27922
Google Scholar
[31] Xu H, Xu W, Wang S, Wu S 2018 J. Opt. 20 095702
Google Scholar
[32] Zhu Y, Xu W, Shi Y 2019 Opt. Commun. 435 426
Google Scholar
[33] Zheng G, Horstmeyer R, Yang C 2013 Nat. Photonics 7 739
Google Scholar
[34] Chen S, Xu T, Zhang J, Wang X, Zhang Y 2018 IEEE Access 6 33399
Google Scholar
[35] Gupta S, Channappayya S S 2019 2019 53rd Asilomar Conf. Signals Syst. Comput Pacific Grove, CA, USA, November, 2019 pp1267–1271
[36] Sun Y, Xu S, Li Y, Tian L, Wohlberg B, Kamilov U S 2019 ICASSP 2019-2019 IEEE Int. Conf. Acoust. Speech Signal Process ICASSP, Brighton, United Kingdom, May, 2019 pp7665–7669
[37] Maiden A M, Humphry M J, Rodenburg J M 2012 J. Opt. Soc. Am. A 29 1606
Google Scholar
[38] Barutcu S, Ruiz P, Schiffers F, Aslan S, Gursoy D, Cossairt O, Katsaggelos A K 2020 2020 IEEE Int. Conf. Image Process. ICIP Abu Dhabi, United Arab Emirates, October, 2020 pp96–100
[39] Tsai E H, Billaud J, Sanchez D F, Ihli J, Odstrčil M, Holler M, Grolimund D, Villevieille C, Guizar-Sicairos M 2019 IScience 11 356
Google Scholar
[40] Zhang Z, Khong J C, Koe B, Luo S, Huang S, Qin L, Cipiccia S, Batey D, Bodey A J, Rau C, Chiu Y L, Zhang Z, Gebelin J C, Green N, Mi J 2021 Scr. Mater. 193 71
Google Scholar
[41] Chamard V, Allain M, Godard P, Talneau A, Patriarche G, Burghammer M 2015 Sci. Rep. 5 1
[42] Chang C, Pan X, Tao H, Liu C, Veetil S P, Zhu J 2021 Opt. Express 29 30878
Google Scholar
[43] Hüe F, Rodenburg J M, Maiden A M, Midgley P A 2011 Ultramicroscopy 111 1117
Google Scholar
[44] Paganin D, Nugent K A 1998 Phys. Rev. Lett. 80 2586
Google Scholar
[45] Whitehead L W, Williams G J, Quiney H M, Vine D J, Dilanian R A, Flewett S, Nugent K A, Peele A G, Balaur E, McNulty I 2009 Phys. Rev. Lett. 103 243902
Google Scholar
[46] Flewett S, Quiney H M, Tran C Q, Nugent K A 2009 Opt. Lett. 34 2198
Google Scholar
[47] Thibault P, Menzel A 2013 Nature 494 68
Google Scholar
[48] Odstrcil M, Baksh P, Boden S A, Card R, Chad J E, Frey J G, Brocklesby W S 2016 Opt. Express 24 8360
Google Scholar
[49] Chen Z, Odstrcil M, Jiang Y, Han Y, Chiu M H, Li L J, Muller D A 2020 Nat. Commun. 11 2994
Google Scholar
[50] Mandula O, Elzo Aizarna M, Eymery J, Burghammer M, Favre-Nicolin V 2016 J. Appl. Crystallogr. 49 1842
Google Scholar
[51] Enders B, Thibault P 2016 Proc. R. Soc. Math. Phys. Eng. Sci. 472 20160640
Google Scholar
[52] Yue K, Deng J, Jiang Y, Nashed Y, Vine D, Vogt S 2021 X-Ray Nanoimaging Instrum. Methods V San Diego, United States, September 8, 2021 p4
[53] Maiden A M, Humphry M J, Sarahan M C, Kraus B, Rodenburg J M 2012 Ultramicroscopy 120 64
Google Scholar
[54] Zhang F, Peterson I, Vila-Comamala J, Diaz A, Berenguer F, Bean R, Chen B, Menzel A, Robinson I K, Rodenburg J M 2013 Opt. Express 21 13592
Google Scholar
[55] El-Gohary M, McNames J 2007 IEEE Trans. Biomed. Eng. 54 2214
Google Scholar
[56] 贾佳 2021 科学观察 16 31
Google Scholar
Jia J 2021 Science Focus 16 31
Google Scholar
[57] Maiden A, Johnson D, Li P 2017 Optica 4 736
Google Scholar
[58] Kappeler A, Ghosh S, Holloway J, Cossairt O, Katsaggelos A 2017 2017 IEEE Int. Conf. Image Process ICIP Beijing, September, 2017 pp1712–1716
[59] Holloway J, Asif M S, Sharma M K, Matsuda N, Horstmeyer R, Cossairt O, Veeraraghavan A 2016 IEEE Trans. Comput. Imaging 2 251
Google Scholar
[60] Nguyen T, Xue Y, Li Y, Tian L, Nehmetallah G 2018 Opt. Express 26 26470
Google Scholar
[61] Chen Y, Luo Z, Wu X, Yang H, Huang B 2020 arXiv: 2003.07460 [eess. IV]
[62] Metzler C A, Schniter P, Veeraraghavan A, Baraniuk R G 2018 arXiv: 1803.00212 [stat. ML]
[63] Romano Y, Elad M, Milanfar P 2017 SIAM J. Imaging Sci. 10 1804
Google Scholar
[64] Zhang K, Zuo W, Chen Y, Meng D, Zhang L 2017 IEEE Trans. Image Process. 26 3142
Google Scholar
[65] Işıl Ç, Oktem F S, Koç A 2019 Appl. Opt. 58 5422
Google Scholar
[66] Cherukara M J, Zhou T, Nashed Y, Enfedaque P, Hexemer A, Harder R J, Holt M V 2020 arXiv: 2004.08247[eess. IV]
[67] Welker S, Peer T, Chapman H N, Gerkmann T 2022 ICASSP 2022–2022 IEEE Int. Conf. Acoust. Speech Signal Process ICASSP Singapore, Singapore, May 23 pp1591–1595
[68] Wengrowicz O, Peleg O, Zahavy T, Loevsky B, Cohen O 2020 Opt. Express 28 17511
Google Scholar
[69] Zhou M, Bai C, Zhang Y, Li R, Peng T, Qian J, Dan D, Min J, Zhou Y, Yao B 2022 IEEE Photonics Technol. Lett. 34 295
Google Scholar
[70] Yao Y, Chan H, Sankaranarayanan S, Balaprakash P, Harder R J, Cherukara M J 2022 Npj Comput. Mater. 8 1
Google Scholar
[71] Nashed Y S G, Vine D J, Peterka T, Deng J, Ross R, Jacobsen C 2014 Opt. Express 22 32082
Google Scholar
[72] Favre-Nicolin V, Girard G, Leake S, Carnis J, Chushkin Y, Kieffer J, Paleo P, Richard M I 2020 J. Appl. Crystallogr. 53 1404
Google Scholar
[73] Thibault P, Guizar-Sicairos M 2012 New J. Phys. 14 063004
Google Scholar
[74] Marchesini S, Krishnan H, Daurer B J, Shapiro D A, Perciano T, Sethian J A, Maia F R N C 2016 J. Appl. Crystallogr. 49 1245
Google Scholar
[75] Luke D R 2005 Inverse Probl. 21 37
Google Scholar
[76] Wakonig K, Stadler H C, Odstrčil M, Tsai E H, Diaz A, Holler M, Usov I, Raabe J, Menzel A, Guizar-Sicairos M 2020 J. Appl. Crystallogr. 53 574
Google Scholar
[77] OpenMP Architecture Review Board (2011) (OpenMP Application Program Interface)
[78] Odstrčil M, Menzel A, Guizar-Sicairos M 2018 Opt. Express 26 3108
Google Scholar
[79] Dieter W, Anastasiia L, Achim S, et al. 2021 Ptychography 4.0: 0.1.0 (Zenodo)
[80] Pennycook T J, Lupini A R, Yang H, Murfitt M F, Jones L, Nellist P D 2015 Ultramicroscopy 151 160
Google Scholar
[81] 潘兴臣, 刘诚, 肖伟刚, 朱健强 2022 激光与光电子学进展 59 2200001
Google Scholar
Pan X C, Liu C, Xiao W G, Zhu J Q 2022 Laser Optoelectron. Prog. 59 2200001
Google Scholar
[82] Jiang S, Guo C, Song P, Zhou N, Bian Z, Zhu J, Wang R, Dong P, Zhang Z, Liao J, Yao J, Feng B, Murphy M, Zheng G 2021 ACS Photonics 8 3261
Google Scholar
[83] Rong L, Tan F, Wang D, Zhang Y, Li K, Zhao J, Wang Y 2021 Opt. Lasers Eng. 147 106729
Google Scholar
[84] Venkatakrishnan S V, Farmand M, Yu Y S, Majidi H, van Benthem K, Marchesini S, Shapiro D A, Hexemer A 2016 IEEE Signal Process. Lett. 23 944
Google Scholar
[85] Jiang Y, Chen Z, Han Y, Deb P, Gao H, Xie S, Purohit P, Tate M W, Park J, Gruner S M, Elser V, Muller D A 2018 Nature 559 343
Google Scholar
[86] Lo Y H, Zhou J, Rana A, Morrill D, Gentry C, Enders B, Yu Y S, Sun C Y, Shapiro D A, Falcone R W, Kapteyn H C, Murnane M M, Gilbert P U P A, Miao J 2021 Proc. Natl. Acad. Sci. 118 e2019068118
Google Scholar
[87] Zhu X, Hitchcock A P, Bazylinski D A, Denes P, Joseph J, Lins U, Marchesini S, Shiu H W, Tyliszczak T, Shapiro D A 2016 Proc. Natl. Acad. Sci. 113 E8219
Google Scholar
[88] Zhou L, Song J, Kim J S, Pei X, Huang C, Boyce M, Mendonça L, Clare D, Siebert A, Allen C S, Liberti E, Stuart D, Pan X, Nellist P D, Zhang P, Kirkland A I, Wang P 2020 Nat. Commun. 11 2773
Google Scholar
[89] Fernandes M F, Neves L 2019 Sci. Rep. 9 1
Google Scholar
[90] Li P, Maiden A 2018 Sci. Rep. 8 1
[91] Ihli J, Levenstein M A, Kim Y Y, Wakonig K, Ning Y, Tatani A, Kulak A N, Green D C, Holler M, Armes S P 2020 Chem. Sci. 11 355
Google Scholar
[92] Fevola G, Jørgensen P S, Verezhak M, Slyamov A, Crovetto A, Balogh Z I, Rein C, Canulescu S, Andreasen J W 2020 Phys. Rev. Res. 2 013378
Google Scholar
[93] Ihli J, Diaz A, Shu Y, Guizar-Sicairos M, Holler M, Wakonig K, Odstrcil M, Li T, Krumeich F, Müller E 2018 J. Phys. Chem. C 122 22920
Google Scholar
[94] Baier S, Damsgaard C D, Scholz M, Benzi F, Rochet A, Hoppe R, Scherer T, Shi J, Wittstock A, Weinhausen B 2016 Microsc. Microanal. 22 178
[95] Dou W, Zhao X, Yin X, Wang H, Luo Y, Qi L 2020 IEEE Trans. Ind. Inform. 17 2842
Google Scholar
-
图 1 (a) Ptychography的基本光路图; (b) PIE算法对应的算法流程和重建结果[12]; (c) DM算法对应的算法流程和重建结果[21]
Fig. 1. (a) Schematic diagram of the basic optical path of ptychography; (b) the corresponding algorithmic flow and reconstruction results of the PIE algorithms[12]; (c) the corresponding algorithmic flow and reconstruction results of the DM algorithms[21].
图 2 (a) fly-PIE的原理示意图[26]; (b) SSP算法原理示意图[27]; (c) fly-PIE算法单模态和多模态的重建图像对比; (d) SSP实验对强散射样品和蜜蜂翅膀的光斑、模量和相位结果
Fig. 2. (a) Diagram of fly-PIE[26]; (b) diagram of SSP[27]; (c) comparison of reconstructed images of fly-PIE algorithm using single and multi-modal; (d) probe, modulus, and phase results of the SSP experiment for stronger scattered samples and the bee wings.
图 3 (a) 3PIE 算法的示意图[37]; (b) 使用 3D-ptychography技术对部分纸张组织的图像采集结果[37] (图(b)前两幅是上层组织的模量和相位, 后两幅是下层组织的模量和相位)
Fig. 3. (a) Schematic diagram of the 3PIE algorithm[37]; (b) the result of image acquisition of some paper tissues using the 3D-ptychography technique[37] (The first two pictures shows the modulus and phase of the upper tissue, and the last two shows the modulus and phase of the lower tissue in Fig. (b)).
图 4 (a) 多模态产生原因的示意图和多模态重建图[47] (其中模糊的图片为单模态重建，下方为探针的 5 个主要模态); (b) 单层WS2样品在不同电子剂量条件下单模态和多模态的重建图像[49]; (c), (d) 分别使用OPRP算法和ePIE算法的重建结果和探针的4个主要模态占比[48]
Fig. 4. (a) Schematic diagram of the causes of multimodal generation and the multimodal reconstruction[47] (The blurred image is the unimodal reconstruction, and the 5 main modes of the probe are shown below); (b) the reconstructed images of unimodal and multimodal reconstructions of the monolayer WS2 sample under different electron dose conditions[49]; (c), (d) the reconstruction results using OPRP algorithm and ePIE algorithm respectively and the four major mode occupancies of the probe[48]
图 5 位置误差对图像重建质量影响示意图[43] (a) 对应图(b)—(e)的探针位置; (b) 随机位置误差重建的结果; (c) 探针逆时针倾斜重建的结果; (d) 探针漂移重建的结果; (e) 探针位置在垂直方向上有扩展的重建结果
Fig. 5. Diagram of the effect of position error on image reconstruction quality[43]: (a) Probe positions in Fig. (b)–(e); (b) result of the random position error reconstruction; (c) result of counterclockwise tilt of the probe reconstruction; (d) result of the probe drift reconstruction; (e) result of the reconstruction of the probe position with extension in the vertical direction.
图 6 退火算法示意图及其位置修正和重建结果的对比[53], 左图为退火算法与ePIE算法重建对比; 右图为e-pcPIE算法的重建对比, 其中图(c)是ePIE的重建结果, 图(d)是退火算法重建结果, 图(e)是e-pcPIE算法重建结果. 全局修正e-pcPIE算法在重建图像和误差上的表现都比退火算法好
Fig. 6. Schematic diagram of the annealing algorithm and comparison of its position correction and reconstruction results[53]. The left figure shows the reconstruction comparison between annealing algorithm and ePIE algorithm.The right figure shows the reconstruction comparison of e-pcPIE algorithm, where Fig. (c) shows the reconstruction result of ePIE, Fig. (d) shows the reconstruction result of annealing algorithm, and Fig. (e) shows the reconstruction result of e-pcPIE algorithm. The globally corrected e-pcPIE algorithm performs better than the annealing algorithm in both reconstructed images and errors.
图 7 (a), (b) 在每一个扫描位置x和y方向的位置误差[54]; (c), (d) 使用可见光的X射线重建样品图像结果[54]; (e)—(h) 使用调制器的X射线重建样品图像结果[54]; (i)，(j) 使用KB镜的X射线重建样品图像结果[54]. 其中图(c), (e), (g), (i)是使用相关匹配算法优化的结果，图(d), (f), (h), (j)是未使用相关匹配算法优化的结果
Fig. 7. (a), (b) Position errors in the x and y directions at each scan position[54]; (c), (d) X-ray reconstructed sample image results using visible light[54]; (e)—(h) X-ray reconstructed sample image results using modulator[54]; (i), (j) X-ray reconstructed sample image using KB mirror[54]. Fig. (c), (e), (g), (i) and Fig. (d), (f), (h), (j) are the results optimized with and without using the corss-correlation algorithm, respectively.
图 9 (a)—(d) PtychoNN与ePIE重建的结果对比[66] (a), (c) 使用ePIE算法重建的结果; (b), (d) 使用PtychoNN重建的结果. (e), (f) PtychoNN在少量数据集上的重建质量的对比, 发现在至少800个数据集上可以得到合理结果; (g) PtychoNN网络的结构
Fig. 9. The left figure above shows and the right figure shows the comparison of the results of the reconstruction of PtychoNN and ePIE[66]: (a), (c) The results of the reconstruction using the ePIE algorithm; (b), (d) the results of the reconstruction using PtychoNN. (e), (f) Comparison the reconstruction quality of PtychoNN on a small number of datasets, and find that reasonable results can be obtained on at least 800 datasets; (g) structure of the PtychoNN network.
图 10 (a) DIP网络的结构示意图; (b) 使用DIP网络、DM算法与AP 算法重建图像结果的对比[67], 其中DIPcg和DIPvm指两种不同模式的DIP网络. 右上方表格为误差阈值为0.1的情况下各个算法需要的迭代次数和时间, 可以看到DIP网络表现很好. (c) DIP网络运用到MINST库上的测试结果
Fig. 10. (a) Schematic diagram of the structure of DIP network; (b) the comparison of the reconstructed image results using DIP network, DM algorithm and AP algorithm in reconstructed image results[67], where DIPcg and DIPvm refer to two different modes of DIP networks. The table on the top right shows the number of iterations and time required by each algorithm for an error threshold of 0.1, and we can see that the DIP network performs well. (c) Test results of the DIP network applied to the MINST library.
图 11 (a), (b) 分别为GPU异步并行和同步并行的数据处理示意图[71]; (c), (d) 分别为Ptychopy软件的工作流程图和GUI界面[52]
Fig. 11. (a), (b) Schematic diagrams of data processing with GPU asynchronous parallelism and synchronous parallelism, respectively[71]; (c), (d) the workflow diagram and GUI interface of the Ptychopy software, respectively[52].
图 12 (a) 左图为使用Ptychopy在同一数据集上测试不同的迭代次数的软件速度, 右图为使用PyNX测试模拟数据, 分别处理200, 500, 1000, 3000张探针大小为256×256的数据集. 横坐标代表数据的大小, 纵坐标代表算法实现的时间 (每一步DM算法会分批次计算LLK(log-likehood)). (b) SHARP的GPU并行示意图和重建流程图[74]. (c) 左图为PyNX重建样品的部分相位图, 右图为分配给不同 GPU的扫描位置, 部分位置的信息将共享[72]
Fig. 12. (a) The left figure in (a) shows the software speedup using Ptychopy to test different number of iterations on the same dataset, and the right figure shows the simulated data using PyNX to test and process 200, 500, 1000, and 3000 datasets with probe size of 256×256, respectively. The horizontal coordinate represents the size of the data, and the vertical coordinate represents the implementation time of the algorithm (LLK (log-likehood) is computed in batches for each DM algorithm step). (b) GPU parallelism of SHARP and the reconstruction flow chart[74]. (c) The left figure in (c) shows the partial phase map of the PyNX reconstructed sample, and the right figure shows the scan positions assigned to different GPUs, and the information of some positions will be shared[72].
表 1 部分软件计算的硬件信息、使用软件的算法对单次迭代计算时间的估计值[76], 以及使用A100 GPU的性能速度预测值
Table 1. Hardware information for part of the software as well as an estimate of the time for a single iterative computation using the various algorithm[76], and the performance is predicted using the A100 GPU.
Ptycholib/Ptychopy PyNX SHARP PtychoShelves 集群名 Argonne ALCF ESRF cluster — PSI GPU型号 NVIDIA K80/A100 NVIDIA V100 GTX Titan Nvidia V100 算法 ePIE DM RAAR DM 单次迭代时间/ns 9.3/— 2.5 1.4 1.03 Nvidia A100/ns 2.8/— 0.7 0.07 0.3 
下载: 导出CSV
表 2 不同软件的特点以及标准化重建时间[76]
Table 2. Characteristics of different software and standardized reconstruction time[76].
软件 算法支持 GPU并行支持 GUI界面 编程/语言/编译 开源 标准化重建时间[76]/(ns·ite–1) Ptychopy DM, ePIE, LSQML $ \surd $ $ \surd $ Python $ \surd $ 9.3 PyNX DM, AP, ML $ \surd $ — Python $ \surd $ 2.5 SHARP RAAR&独立编写 $ \surd $ $ \surd $ — — 1.4 Ptypy DM, ePIE, ML $ \surd $ — Python — — PtychoShelves DM, ML $ \surd $ — Matlab $ \surd $ 1.03/0.7 Ptychography4.0 独立编写 $ \surd $ — Python $ \surd $ —
下载: 导出CSV
-
[1] Bates R 1982 Opt. Stuttg. 61 5
[2] 赵江涛 2020 博士学位论文 (合肥: 中国科学技术大学)
Zhao J T 2020 Ph. D. Dissertation(Hefei: University of Science and Technology of China) (in Chinese)
[3] Bates R, Fright W R 1983 J. Opt. Soc. Am. 73 358
[4] Miao J, Charalambous P, Kirz J, Sayre D 1999 Nature 400 342
Google Scholar
[5] Gerchberg R W, Saxton W 1971 Optik 35 237
[6] Fienup J R 1978 Opt. Lett. 3 27
Google Scholar
[7] Fienup J R 1982 Appl. Opt. 21 2758
Google Scholar
[8] Fienup J R, Wackerman C C 1986 JOSA A 3 1897
Google Scholar
[9] Rodenburg J M, Faulkner H M L 2004 Appl. Phys. Lett. 85 4795
Google Scholar
[10] Faulkner H M L, Rodenburg J M 2004 Phys. Rev. Lett. 93 023903
Google Scholar
[11] Saxton W O 2013 Computer Techniques for Image Processing in Electron Microscopy (Academic Press) pp78–96
[12] 潘兴臣, 刘诚, 陶华, 刘海岗, 朱健强 2020 光学学报 40 0111010
Google Scholar
Pan X C, Liu C, Tao H, Liu H G, Zhu J Q 2020 Acta Opt. Sin. 40 0111010
Google Scholar
[13] Rodenburg J M, Hurst A C, Cullis A G 2007 Ultramicroscopy 107 227
Google Scholar
[14] Rodenburg J M 2008 Adv. Imaging Electro. Phys. 2008 87
Google Scholar
[15] Moxham T E, Laundy D, Dhamgaye V, Fox O J, Sawhney K, Korsunsky A M 2021 Appl. Phys. Lett. 118 104104
Google Scholar
[16] Shemilt L, Verbanis E, Schwenke J, Estandarte A K, Xiong G, Harder R, Parmar N, Yusuf M, Zhang F, Robinson I K 2015 Biophys. J. 108 706
Google Scholar
[17] Bhartiya A, Batey D, Cipiccia S, Shi X, Rau C, Botchway S, Yusuf M, Robinson I K 2021 Chromosome Res. 29 107
Google Scholar
[18] Beckers M, Senkbeil T, Gorniak T, Reese M, Giewekemeyer K, Gleber S C, Salditt T, Rosenhahn A 2011 Phys. Rev. Lett. 107 208101
Google Scholar
[19] D’alfonso A J, Morgan A J, Yan A W C, Wang P, Sawada H, Kirkland A I, Allen L J 2014 Phys. Rev. B 89 064101
Google Scholar
[20] Kane D J 2019 IEEE J. Sel. Top. Quantum Electron. 25 1
Google Scholar
[21] Thibault P, Dierolf M, Bunk O, Menzel A, Pfeiffer F 2009 Ultramicroscopy 109 338
Google Scholar
[22] Maiden A M, Rodenburg J M 2009 Ultramicroscopy 109 1256
Google Scholar
[23] Bunk O, Dierolf M, Kynde S, Johnson I, Marti O, Pfeiffer F 2008 Ultramicroscopy 108 481
Google Scholar
[24] [25] Dierolf M, Thibault P, Menzel A, Kewish C M, Jefimovs K, Schlichting I, König K von, Bunk O, Pfeiffer F 2010 New J. Phys. 12 035017
Google Scholar
[26] Clark J N, Huang X, Harder R J, Robinson I K 2014 Opt. Lett. 39 6066
Google Scholar
[27] Pan X, Liu C, Zhu J 2013 Appl. Phys. Lett. 103 2758
Google Scholar
[28] Sidorenko P, Cohen O 2016 Optica 3 9
Google Scholar
[29] Chen B K, Sidorenko P, Lahav O, Peleg O, Cohen O 2018 Opt. Lett. 43 5379
Google Scholar
[30] Xu W, Xu H, Luo Y, Li T, Shi Y 2016 Opt. Express 24 27922
Google Scholar
[31] Xu H, Xu W, Wang S, Wu S 2018 J. Opt. 20 095702
Google Scholar
[32] Zhu Y, Xu W, Shi Y 2019 Opt. Commun. 435 426
Google Scholar
[33] Zheng G, Horstmeyer R, Yang C 2013 Nat. Photonics 7 739
Google Scholar
[34] Chen S, Xu T, Zhang J, Wang X, Zhang Y 2018 IEEE Access 6 33399
Google Scholar
[35] Gupta S, Channappayya S S 2019 2019 53rd Asilomar Conf. Signals Syst. Comput Pacific Grove, CA, USA, November, 2019 pp1267–1271
[36] Sun Y, Xu S, Li Y, Tian L, Wohlberg B, Kamilov U S 2019 ICASSP 2019-2019 IEEE Int. Conf. Acoust. Speech Signal Process ICASSP, Brighton, United Kingdom, May, 2019 pp7665–7669
[37] Maiden A M, Humphry M J, Rodenburg J M 2012 J. Opt. Soc. Am. A 29 1606
Google Scholar
[38] Barutcu S, Ruiz P, Schiffers F, Aslan S, Gursoy D, Cossairt O, Katsaggelos A K 2020 2020 IEEE Int. Conf. Image Process. ICIP Abu Dhabi, United Arab Emirates, October, 2020 pp96–100
[39] Tsai E H, Billaud J, Sanchez D F, Ihli J, Odstrčil M, Holler M, Grolimund D, Villevieille C, Guizar-Sicairos M 2019 IScience 11 356
Google Scholar
[40] Zhang Z, Khong J C, Koe B, Luo S, Huang S, Qin L, Cipiccia S, Batey D, Bodey A J, Rau C, Chiu Y L, Zhang Z, Gebelin J C, Green N, Mi J 2021 Scr. Mater. 193 71
Google Scholar
[41] Chamard V, Allain M, Godard P, Talneau A, Patriarche G, Burghammer M 2015 Sci. Rep. 5 1
[42] Chang C, Pan X, Tao H, Liu C, Veetil S P, Zhu J 2021 Opt. Express 29 30878
Google Scholar
[43] Hüe F, Rodenburg J M, Maiden A M, Midgley P A 2011 Ultramicroscopy 111 1117
Google Scholar
[44] Paganin D, Nugent K A 1998 Phys. Rev. Lett. 80 2586
Google Scholar
[45] Whitehead L W, Williams G J, Quiney H M, Vine D J, Dilanian R A, Flewett S, Nugent K A, Peele A G, Balaur E, McNulty I 2009 Phys. Rev. Lett. 103 243902
Google Scholar
[46] Flewett S, Quiney H M, Tran C Q, Nugent K A 2009 Opt. Lett. 34 2198
Google Scholar
[47] Thibault P, Menzel A 2013 Nature 494 68
Google Scholar
[48] Odstrcil M, Baksh P, Boden S A, Card R, Chad J E, Frey J G, Brocklesby W S 2016 Opt. Express 24 8360
Google Scholar
[49] Chen Z, Odstrcil M, Jiang Y, Han Y, Chiu M H, Li L J, Muller D A 2020 Nat. Commun. 11 2994
Google Scholar
[50] Mandula O, Elzo Aizarna M, Eymery J, Burghammer M, Favre-Nicolin V 2016 J. Appl. Crystallogr. 49 1842
Google Scholar
[51] Enders B, Thibault P 2016 Proc. R. Soc. Math. Phys. Eng. Sci. 472 20160640
Google Scholar
[52] Yue K, Deng J, Jiang Y, Nashed Y, Vine D, Vogt S 2021 X-Ray Nanoimaging Instrum. Methods V San Diego, United States, September 8, 2021 p4
[53] Maiden A M, Humphry M J, Sarahan M C, Kraus B, Rodenburg J M 2012 Ultramicroscopy 120 64
Google Scholar
[54] Zhang F, Peterson I, Vila-Comamala J, Diaz A, Berenguer F, Bean R, Chen B, Menzel A, Robinson I K, Rodenburg J M 2013 Opt. Express 21 13592
Google Scholar
[55] El-Gohary M, McNames J 2007 IEEE Trans. Biomed. Eng. 54 2214
Google Scholar
[56] 贾佳 2021 科学观察 16 31
Google Scholar
Jia J 2021 Science Focus 16 31
Google Scholar
[57] Maiden A, Johnson D, Li P 2017 Optica 4 736
Google Scholar
[58] Kappeler A, Ghosh S, Holloway J, Cossairt O, Katsaggelos A 2017 2017 IEEE Int. Conf. Image Process ICIP Beijing, September, 2017 pp1712–1716
[59] Holloway J, Asif M S, Sharma M K, Matsuda N, Horstmeyer R, Cossairt O, Veeraraghavan A 2016 IEEE Trans. Comput. Imaging 2 251
Google Scholar
[60] Nguyen T, Xue Y, Li Y, Tian L, Nehmetallah G 2018 Opt. Express 26 26470
Google Scholar
[61] Chen Y, Luo Z, Wu X, Yang H, Huang B 2020 arXiv: 2003.07460 [eess. IV]
[62] Metzler C A, Schniter P, Veeraraghavan A, Baraniuk R G 2018 arXiv: 1803.00212 [stat. ML]
[63] Romano Y, Elad M, Milanfar P 2017 SIAM J. Imaging Sci. 10 1804
Google Scholar
[64] Zhang K, Zuo W, Chen Y, Meng D, Zhang L 2017 IEEE Trans. Image Process. 26 3142
Google Scholar
[65] Işıl Ç, Oktem F S, Koç A 2019 Appl. Opt. 58 5422
Google Scholar
[66] Cherukara M J, Zhou T, Nashed Y, Enfedaque P, Hexemer A, Harder R J, Holt M V 2020 arXiv: 2004.08247[eess. IV]
[67] Welker S, Peer T, Chapman H N, Gerkmann T 2022 ICASSP 2022–2022 IEEE Int. Conf. Acoust. Speech Signal Process ICASSP Singapore, Singapore, May 23 pp1591–1595
[68] Wengrowicz O, Peleg O, Zahavy T, Loevsky B, Cohen O 2020 Opt. Express 28 17511
Google Scholar
[69] Zhou M, Bai C, Zhang Y, Li R, Peng T, Qian J, Dan D, Min J, Zhou Y, Yao B 2022 IEEE Photonics Technol. Lett. 34 295
Google Scholar
[70] Yao Y, Chan H, Sankaranarayanan S, Balaprakash P, Harder R J, Cherukara M J 2022 Npj Comput. Mater. 8 1
Google Scholar
[71] Nashed Y S G, Vine D J, Peterka T, Deng J, Ross R, Jacobsen C 2014 Opt. Express 22 32082
Google Scholar
[72] Favre-Nicolin V, Girard G, Leake S, Carnis J, Chushkin Y, Kieffer J, Paleo P, Richard M I 2020 J. Appl. Crystallogr. 53 1404
Google Scholar
[73] Thibault P, Guizar-Sicairos M 2012 New J. Phys. 14 063004
Google Scholar
[74] Marchesini S, Krishnan H, Daurer B J, Shapiro D A, Perciano T, Sethian J A, Maia F R N C 2016 J. Appl. Crystallogr. 49 1245
Google Scholar
[75] Luke D R 2005 Inverse Probl. 21 37
Google Scholar
[76] Wakonig K, Stadler H C, Odstrčil M, Tsai E H, Diaz A, Holler M, Usov I, Raabe J, Menzel A, Guizar-Sicairos M 2020 J. Appl. Crystallogr. 53 574
Google Scholar
[77] OpenMP Architecture Review Board (2011) (OpenMP Application Program Interface)
[78] Odstrčil M, Menzel A, Guizar-Sicairos M 2018 Opt. Express 26 3108
Google Scholar
[79] Dieter W, Anastasiia L, Achim S, et al. 2021 Ptychography 4.0: 0.1.0 (Zenodo)
[80] Pennycook T J, Lupini A R, Yang H, Murfitt M F, Jones L, Nellist P D 2015 Ultramicroscopy 151 160
Google Scholar
[81] 潘兴臣, 刘诚, 肖伟刚, 朱健强 2022 激光与光电子学进展 59 2200001
Google Scholar
Pan X C, Liu C, Xiao W G, Zhu J Q 2022 Laser Optoelectron. Prog. 59 2200001
Google Scholar
[82] Jiang S, Guo C, Song P, Zhou N, Bian Z, Zhu J, Wang R, Dong P, Zhang Z, Liao J, Yao J, Feng B, Murphy M, Zheng G 2021 ACS Photonics 8 3261
Google Scholar
[83] Rong L, Tan F, Wang D, Zhang Y, Li K, Zhao J, Wang Y 2021 Opt. Lasers Eng. 147 106729
Google Scholar
[84] Venkatakrishnan S V, Farmand M, Yu Y S, Majidi H, van Benthem K, Marchesini S, Shapiro D A, Hexemer A 2016 IEEE Signal Process. Lett. 23 944
Google Scholar
[85] Jiang Y, Chen Z, Han Y, Deb P, Gao H, Xie S, Purohit P, Tate M W, Park J, Gruner S M, Elser V, Muller D A 2018 Nature 559 343
Google Scholar
[86] Lo Y H, Zhou J, Rana A, Morrill D, Gentry C, Enders B, Yu Y S, Sun C Y, Shapiro D A, Falcone R W, Kapteyn H C, Murnane M M, Gilbert P U P A, Miao J 2021 Proc. Natl. Acad. Sci. 118 e2019068118
Google Scholar
[87] Zhu X, Hitchcock A P, Bazylinski D A, Denes P, Joseph J, Lins U, Marchesini S, Shiu H W, Tyliszczak T, Shapiro D A 2016 Proc. Natl. Acad. Sci. 113 E8219
Google Scholar
[88] Zhou L, Song J, Kim J S, Pei X, Huang C, Boyce M, Mendonça L, Clare D, Siebert A, Allen C S, Liberti E, Stuart D, Pan X, Nellist P D, Zhang P, Kirkland A I, Wang P 2020 Nat. Commun. 11 2773
Google Scholar
[89] Fernandes M F, Neves L 2019 Sci. Rep. 9 1
Google Scholar
[90] Li P, Maiden A 2018 Sci. Rep. 8 1
[91] Ihli J, Levenstein M A, Kim Y Y, Wakonig K, Ning Y, Tatani A, Kulak A N, Green D C, Holler M, Armes S P 2020 Chem. Sci. 11 355
Google Scholar
[92] Fevola G, Jørgensen P S, Verezhak M, Slyamov A, Crovetto A, Balogh Z I, Rein C, Canulescu S, Andreasen J W 2020 Phys. Rev. Res. 2 013378
Google Scholar
[93] Ihli J, Diaz A, Shu Y, Guizar-Sicairos M, Holler M, Wakonig K, Odstrcil M, Li T, Krumeich F, Müller E 2018 J. Phys. Chem. C 122 22920
Google Scholar
[94] Baier S, Damsgaard C D, Scholz M, Benzi F, Rochet A, Hoppe R, Scherer T, Shi J, Wittstock A, Weinhausen B 2016 Microsc. Microanal. 22 178
[95] Dou W, Zhao X, Yin X, Wang H, Luo Y, Qi L 2020 IEEE Trans. Ind. Inform. 17 2842
Google Scholar
下载:
计量
- 文章访问数: 9761
- PDF下载量: 511
- 被引次数: 0
