搜索

x

留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

基于YOLOv3框架的高分辨电镜图像原子峰位置检测

南虎 麻晓晶 赵海博 汤少杰 刘卫华 王大威 贾春林

引用本文:
Citation:

基于YOLOv3框架的高分辨电镜图像原子峰位置检测

南虎, 麻晓晶, 赵海博, 汤少杰, 刘卫华, 王大威, 贾春林

Detection of intensity peaks in high-resolution transmission electron microscopy image based on YOLOv3

Nan Hu, Ma Xiao-Jing, Zhao Hai-Bo, Tang Shao-Jie, Liu Wei-Hua, Wang Da-Wei, Jia Chun-Lin
PDF
HTML
导出引用
  • 高分辨电镜图像中原子峰位置的检测具有十分重要的现实意义, 通过精确定量化原子峰位置可以分析物质在微观尺度上的结构形变、电极化矢量分布等重要信息. 近年来深度学习技术在图像目标检测领域取得了巨大突破, 这一技术可用在高分辨电镜图像处理上,因为原子位置的检测可以看作是一个目标检测问题. 本文利用先进的机器学习方法, 通过制作高质量原子图像样本集, 使用YOLOv3目标识别框架对原子图像进行自动检测, 达到预期效果, 实现了深度学习技术在高分辨电镜图像处理领域的应用. 该方法的运用有望突破自动处理动态、大量电镜图片的瓶颈问题.
    The detection of intensity peaks, which correspond to atom positions, in high-resolution (scanning) transmission electron microscopy images is of great practical significance. By quantitatively determining the locations of these peaks, it is possible to obtain important information such as the structural deformation and the electric dipole distribution inside a material on the nanoscale. The detection of the peak positions in image processing can be regarded as a target detection problem, for which breakthroughs have been made with deep-learning neural networks. Comparing to the traditional target detection algorithms, which are based on specifically designed feature extractor and classifier, the deep-learning approach can obtain the features at multiple levels of abstraction automatically, thus improving the robustness of the detection process. In this paper, we realize the automatic detection of the intensity peaks in high-resolution electron microscopy images by building a high-quality atomic image sample set and using the YOLOv3 target detection framework. With its accuracy and speed, which are superior over other target detection neural networks, the YOLOv3 is suitable for image processing as the number of images increases explosively. The YOLOv3 network converges well in the training process using our atomic image sample set, with the loss function reaching a minimum after 500 epoches; the trained neural network can detect almost all the major atoms in the images, showing its excellent ability. With the aid of YOLOv3, we also develop a program to organize the detected atoms, enabling the detection of all the other atoms within each unit cell. It is found that, combining YOLOv3 with the newly developed algorithm, almost all the atoms can be successfully determined, showing its advantages over previous algorithms based on bravis lattice construction, especially for high-resolution transmission electron microscopy images with lattice defects. Our results show that this image processing technique has the potential in overcoming the bottleneck in the fast processing of high resolution electron microscopy images.
      通信作者: 王大威, dawei.wang@xjtu.edu.cn
    • 基金项目: 国家自然科学基金(批准号: 11974268, 61671368)和陕西省重点研发计划(批准号: 2020SF-377)资助的课题
      Corresponding author: Wang Da-Wei, dawei.wang@xjtu.edu.cn
    • Funds: National Natural Science Foundation of China (Grant Nos. 11974268, 61671368) and the Key R&D Program of Shaanxi Province, China (Grant No. 2020SF-377)
    [1]

    Urban K W 2008 Science 321 506Google Scholar

    [2]

    Falke U, Bleloch A, Falke M, Teichert S 2004 Phys. Rev. Lett. 92 116103Google Scholar

    [3]

    Nellist P D, Chisholm M F, Dellby N, Krivanek O L, Murfitt M F, Szilagyi Z S, Lupini A R, Borisevich A, Sides W H, Pennycook S J 2004 Science 305 1741Google Scholar

    [4]

    Jia C L, Urban K 2004 Science 303 2001Google Scholar

    [5]

    Mkhoyan K A, Batson P E, Cha J, Schaff W J, Silcox J 2006 Science 312 1354Google Scholar

    [6]

    Varela M, Findlay S D, Lupini A R, Christen H M, Borisevich A Y, Dellby N, Krivanek O L, Nellist P D, Oxley M P, Allen L J, Pennycook S J 2004 Phys. Rev. Lett. 92 095502Google Scholar

    [7]

    Bosman M, Keast V J, Garcia-Munoz J L, D'Alfonso A J, Findlay S D, Allen L J 2007 Phys. Rev. Lett. 99 086102Google Scholar

    [8]

    Muller D A, Kourkoutis L F, Murfitt M, Song J H, Hwang H Y, Silcox J, Dellby N, Krivanek O L 2008 Science 319 1073Google Scholar

    [9]

    Scott J F 2007 Science 315 954Google Scholar

    [10]

    Nakagawa N, Hwang H Y, Muller D A 2006 Nat. Mater. 5 204Google Scholar

    [11]

    Reyren N, Thiel S, Caviglia A D, Kourkoutis L F, Hammerl G, Richter C, Schneider C W, Kopp T, Ruetschi A S, Jaccard D, Gabay M, Muller D A, Triscone J M, Mannhart J 2007 Science 317 1196Google Scholar

    [12]

    Jia C L, Urban K W, Alexe M, Hesse D, Vrejoiu I 2011 Science 331 1420Google Scholar

    [13]

    Lu L, Nahas Y, Liu M, Du H, Jiang Z, Ren S, Wang D, Jin L, Prokhorenko S, Jia C L, Bellaiche L 2018 Phys. Rev. Lett. 120 177601Google Scholar

    [14]

    Tang Y L, Zhu Y L, Ma X L, Borisevich A Y, Morozovska A N, Eliseev E A, Wang W Y, Wang Y J, Xu Y B, Zhang Z D, Pennycook S J 2015 Science 348 547Google Scholar

    [15]

    Nelson C T, Winchester B, Zhang Y, Kim S J, Melville A, Adamo C, Folkman C M, Baek S H, Eom C B, Schlom D G, Chen L Q, Pan X Q 2011 Nano Lett. 11 828Google Scholar

    [16]

    Catalan G, Lubk A, Vlooswijk A H G, Snoeck E, Magen C, Janssens A, Rispens G, Rijnders G, Blank D H A, Noheda B 2011 Nat. Mater. 10 963Google Scholar

    [17]

    Li S, Wang Y J, Zhu Y L, Tang Y L, Liu Y, Ma J Y, Han M J, Wu B, Ma X L 2019 Acta Mater. 171 176Google Scholar

    [18]

    Gao P, Kumamoto A, Ishikawa R, Lugg N, Shibata N, Ikuhara Y 2018 Ultramicroscopy 184 177Google Scholar

    [19]

    Sun Y, Abid A Y, Tan C, Ren C, Li M, Li N, Chen P, Li Y, Zhang J, Zhong X, Wang J, Liao M, Liu K, Bai X, Zhou Y, Yu D, Gao P 2019 Sci. Adv. 5 eaav4355Google Scholar

    [20]

    Chen P, Zhong X, Zorn J A, Li M, Sun Y, Abid A Y, Ren C, Li Y, Li X, Ma X, Wang J, Liu K, Xu Z, Tan C, Chen L, Gao P, Bai X 2020 Nat. Commun. 11 1840Google Scholar

    [21]

    Du H, Jia C J, Mayer J 2016 Chem. Mater. 28 650Google Scholar

    [22]

    Galindo P L, Kret S, Sanchez A M, Laval J Y, Yanez A, Pizarro J, Guerrero E, Ben T, Molina S I 2007 Ultramicroscopy 107 1186Google Scholar

    [23]

    南虎, 卢江波, 刘明, 井红梅, 汤少杰, 王大威, 贾春林 2016 电子显微学报 035 191Google Scholar

    Nan H, Lu J B, Liu M, Jing H M, Tang S J, Wang D W, Jia C L 2016 J. Chin. Electron. Microsc. Soc. 035 191Google Scholar

    [24]

    Gong Y J, Liu Z, Lupini A R, Shi G, Lin J H, Najmaei S, Lin Z, Elias A L, Berkdemir A, You G, Terrones H, Terrones M, Vajtai R, Pantelides S T, Pennycook S J, Lou J, Zhou W, Ajayan P M 2014 Nano Lett. 14 442Google Scholar

    [25]

    Taigman Y, Yang M, Ranzato M, Wolf L 2014 Proc CVPR IEEE 1701Google Scholar

    [26]

    LeCun Y, Bengio Y, Hinton G 2015 Nature 521 436Google Scholar

    [27]

    Redmon J, Farhadi A 2018 arXiv: 1804.02767 [cs.CV]

    [28]

    Tzutalin. LabelImg. Git code (2015). https://github.com/tzutalin/labelImglabelImg [2020-12-3]

  • 图 1  样本图像标记示例 (a), (b), (d)包含大尺寸原子峰, 做到了100%标记; (c)包含小尺寸原子峰, 做到90%左右的标记.

    Fig. 1.  Sample images with the intensity peaks labeled. The images (a), (b) and (d) contain a relatively small number of atom peaks, which are all labeled. On the other hand, (c) has too many atom peaks, which are partially labeled (about 90%).

    图 2  最大训练批次为3000时网络的训练情况 (a)网络训练过程的误差函数, 在500批次前后收敛, 直至训练结束一直保持稳定; (b)网络训练过程的交并比数值, 在500批次前快速上升, 之后一直缓慢上升, 没有下降趋势

    Fig. 2.  The YOLOv3 training process with an epoch number of 3000: (a) The loss function converges after 500 epochs, which remains stable until the end of the training; (b) the IOU value increases quickly during the first 500 epochs, and then slowly increases until the end of the training.

    图 3  设置置信度为0.25时YOLOv3网络检测结果

    Fig. 3.  The detection results obtained with a confidence threshold of 0.25.

    图 4  设置置信度为0.1的YOLOv3网络检测结果, 各图像的检测时间如下: (a) 7.40 s; (b) 7.38 s; (c) 7.2 s; (d) 7.24 s; (e) 7.28 s; (f) 7.25 s; (g) 7.24 s; (h) 7.24 s; (i) 7.22 s

    Fig. 4.  The detection results obtained with a confidence threshold of 0.1. The detection times for these images are: (a) 7.40 s; (b) 7.38 s; (c) 7.2 s; (d) 7.24 s; (e) 7.28 s; (f) 7.25 s; (g) 7.24 s; (h) 7.24 s; (i) 7.22 s.

    图 5  (a), (b) 1024 × 1024尺寸图像检测效果; (c) 512 × 512尺寸晶格缺陷图像检测效果, 图(a)—图(c)检测时间均在7.4 s左右; (d), (e), (f)分别是(a), (b), (c)中原子目标检测框内二维高斯拟合结果、单胞构建效果以及B位原子峰的拟合结果, 计算时间分别为27.32 s, 10.51 s, 20.52 s

    Fig. 5.  (a), (b)The detection result of a 1024 × 1024 image; (c) the detection result of a 512 × 512 lattice distortion image. The detection time for each of them is about 7.4 s; (d), (e), (f) two-dimensional gauss fitting result within each detected window in (a), (b) and (c), constructed frames of unit cells, and fitting result of corresponding B site atom peaks. Computing times are 27.32 s, 10.51 s, 20.52 s respectively.

  • [1]

    Urban K W 2008 Science 321 506Google Scholar

    [2]

    Falke U, Bleloch A, Falke M, Teichert S 2004 Phys. Rev. Lett. 92 116103Google Scholar

    [3]

    Nellist P D, Chisholm M F, Dellby N, Krivanek O L, Murfitt M F, Szilagyi Z S, Lupini A R, Borisevich A, Sides W H, Pennycook S J 2004 Science 305 1741Google Scholar

    [4]

    Jia C L, Urban K 2004 Science 303 2001Google Scholar

    [5]

    Mkhoyan K A, Batson P E, Cha J, Schaff W J, Silcox J 2006 Science 312 1354Google Scholar

    [6]

    Varela M, Findlay S D, Lupini A R, Christen H M, Borisevich A Y, Dellby N, Krivanek O L, Nellist P D, Oxley M P, Allen L J, Pennycook S J 2004 Phys. Rev. Lett. 92 095502Google Scholar

    [7]

    Bosman M, Keast V J, Garcia-Munoz J L, D'Alfonso A J, Findlay S D, Allen L J 2007 Phys. Rev. Lett. 99 086102Google Scholar

    [8]

    Muller D A, Kourkoutis L F, Murfitt M, Song J H, Hwang H Y, Silcox J, Dellby N, Krivanek O L 2008 Science 319 1073Google Scholar

    [9]

    Scott J F 2007 Science 315 954Google Scholar

    [10]

    Nakagawa N, Hwang H Y, Muller D A 2006 Nat. Mater. 5 204Google Scholar

    [11]

    Reyren N, Thiel S, Caviglia A D, Kourkoutis L F, Hammerl G, Richter C, Schneider C W, Kopp T, Ruetschi A S, Jaccard D, Gabay M, Muller D A, Triscone J M, Mannhart J 2007 Science 317 1196Google Scholar

    [12]

    Jia C L, Urban K W, Alexe M, Hesse D, Vrejoiu I 2011 Science 331 1420Google Scholar

    [13]

    Lu L, Nahas Y, Liu M, Du H, Jiang Z, Ren S, Wang D, Jin L, Prokhorenko S, Jia C L, Bellaiche L 2018 Phys. Rev. Lett. 120 177601Google Scholar

    [14]

    Tang Y L, Zhu Y L, Ma X L, Borisevich A Y, Morozovska A N, Eliseev E A, Wang W Y, Wang Y J, Xu Y B, Zhang Z D, Pennycook S J 2015 Science 348 547Google Scholar

    [15]

    Nelson C T, Winchester B, Zhang Y, Kim S J, Melville A, Adamo C, Folkman C M, Baek S H, Eom C B, Schlom D G, Chen L Q, Pan X Q 2011 Nano Lett. 11 828Google Scholar

    [16]

    Catalan G, Lubk A, Vlooswijk A H G, Snoeck E, Magen C, Janssens A, Rispens G, Rijnders G, Blank D H A, Noheda B 2011 Nat. Mater. 10 963Google Scholar

    [17]

    Li S, Wang Y J, Zhu Y L, Tang Y L, Liu Y, Ma J Y, Han M J, Wu B, Ma X L 2019 Acta Mater. 171 176Google Scholar

    [18]

    Gao P, Kumamoto A, Ishikawa R, Lugg N, Shibata N, Ikuhara Y 2018 Ultramicroscopy 184 177Google Scholar

    [19]

    Sun Y, Abid A Y, Tan C, Ren C, Li M, Li N, Chen P, Li Y, Zhang J, Zhong X, Wang J, Liao M, Liu K, Bai X, Zhou Y, Yu D, Gao P 2019 Sci. Adv. 5 eaav4355Google Scholar

    [20]

    Chen P, Zhong X, Zorn J A, Li M, Sun Y, Abid A Y, Ren C, Li Y, Li X, Ma X, Wang J, Liu K, Xu Z, Tan C, Chen L, Gao P, Bai X 2020 Nat. Commun. 11 1840Google Scholar

    [21]

    Du H, Jia C J, Mayer J 2016 Chem. Mater. 28 650Google Scholar

    [22]

    Galindo P L, Kret S, Sanchez A M, Laval J Y, Yanez A, Pizarro J, Guerrero E, Ben T, Molina S I 2007 Ultramicroscopy 107 1186Google Scholar

    [23]

    南虎, 卢江波, 刘明, 井红梅, 汤少杰, 王大威, 贾春林 2016 电子显微学报 035 191Google Scholar

    Nan H, Lu J B, Liu M, Jing H M, Tang S J, Wang D W, Jia C L 2016 J. Chin. Electron. Microsc. Soc. 035 191Google Scholar

    [24]

    Gong Y J, Liu Z, Lupini A R, Shi G, Lin J H, Najmaei S, Lin Z, Elias A L, Berkdemir A, You G, Terrones H, Terrones M, Vajtai R, Pantelides S T, Pennycook S J, Lou J, Zhou W, Ajayan P M 2014 Nano Lett. 14 442Google Scholar

    [25]

    Taigman Y, Yang M, Ranzato M, Wolf L 2014 Proc CVPR IEEE 1701Google Scholar

    [26]

    LeCun Y, Bengio Y, Hinton G 2015 Nature 521 436Google Scholar

    [27]

    Redmon J, Farhadi A 2018 arXiv: 1804.02767 [cs.CV]

    [28]

    Tzutalin. LabelImg. Git code (2015). https://github.com/tzutalin/labelImglabelImg [2020-12-3]

  • [1] 张赞, 黄北举, 陈弘达. 基于可重构硅光滤波器的计算重建片上光谱仪. 物理学报, 2024, 73(14): 140701. doi: 10.7498/aps.73.20240224
    [2] 施岳, 欧攀, 郑明, 邰含旭, 王玉红, 段若楠, 吴坚. 基于轻量残差复合增强收敛神经网络的粒子场计算层析成像伪影噪声抑制. 物理学报, 2024, 73(10): 104202. doi: 10.7498/aps.73.20231902
    [3] 黄宇航, 陈理想. 基于未训练神经网络的分数傅里叶变换成像. 物理学报, 2024, 73(9): 094201. doi: 10.7498/aps.73.20240050
    [4] 刘鸿江, 刘逸飞, 谷付星. 基于深度学习的微纳光纤自动制备系统. 物理学报, 2024, 73(10): 104207. doi: 10.7498/aps.73.20240171
    [5] 张普渡, 王伟权, 李哲民, 张资旋, 王叶晨, 周泓宇, 银燕. 基于深度学习的钻孔辐射压离子加速建模. 物理学报, 2023, 72(18): 184103. doi: 10.7498/aps.72.20230702
    [6] 刘栋, 崔新月, 王浩东, 张贵军. 蛋白质结构模型质量评估方法综述. 物理学报, 2023, 72(24): 248702. doi: 10.7498/aps.72.20231071
    [7] 欧秀娟, 肖奕. RNA扭转角预测的深度学习方法. 物理学报, 2023, 72(24): 248703. doi: 10.7498/aps.72.20231069
    [8] 孙涛, 袁健美. 基于深度学习原子特征表示方法的Janus过渡金属硫化物带隙预测. 物理学报, 2023, 72(2): 028901. doi: 10.7498/aps.72.20221374
    [9] 李志强, 谭晓瑜, 段忻磊, 张敬义, 杨家跃. 氮化硅微波高温介电函数深度学习分子动力学模拟. 物理学报, 2022, 71(24): 247803. doi: 10.7498/aps.71.20221002
    [10] 向鹏程, 蔡聪波, 王杰超, 蔡淑惠, 陈忠. 基于深度神经网络的时空编码磁共振成像超分辨率重建方法. 物理学报, 2022, 71(5): 058702. doi: 10.7498/aps.71.20211754
    [11] 朱琦, 许多, 张元军, 李玉娟, 王文, 张海燕. 基于卷积神经网络的白蚀缺陷超声探测. 物理学报, 2022, 71(24): 244301. doi: 10.7498/aps.71.20221504
    [12] 战庆亮, 葛耀君, 白春锦. 基于深度学习的流场时程特征提取模型. 物理学报, 2022, 71(7): 074701. doi: 10.7498/aps.71.20211373
    [13] 张航瑛, 王雪琦, 王华英, 曹良才. 基于明度分量的Retinex-Net图像增强改进方法. 物理学报, 2022, 71(11): 110701. doi: 10.7498/aps.71.20220099
    [14] 徐昭, 周昕, 白星, 李聪, 陈洁, 倪洋. 基于深度学习的相位截断傅里叶变换非对称加密系统攻击方法. 物理学报, 2021, 70(14): 144202. doi: 10.7498/aps.70.20202075
    [15] 王甜甜, 王慧, 朱艳春, 王丽嘉. 基于位移流U-Net和变分自动编码器的心脏电影磁共振图像左心肌运动追踪. 物理学报, 2021, 70(22): 228701. doi: 10.7498/aps.70.20210885
    [16] 苏博, 陶芬, 李可, 杜国浩, 张玲, 李中亮, 邓彪, 谢红兰, 肖体乔. 同步辐射纳米CT图像配准方法研究. 物理学报, 2021, 70(16): 160704. doi: 10.7498/aps.70.20210156
    [17] 赵智鹏, 周双, 王兴元. 基于深度学习的新混沌信号及其在图像加密中的应用. 物理学报, 2021, 70(23): 230502. doi: 10.7498/aps.70.20210561
    [18] 张瑶, 张云波, 陈立. 基于深度学习的光学表面杂质检测. 物理学报, 2021, 70(16): 168702. doi: 10.7498/aps.70.20210403
    [19] 郎利影, 陆佳磊, 于娜娜, 席思星, 王雪光, 张雷, 焦小雪. 基于深度学习的联合变换相关器光学图像加密系统去噪方法. 物理学报, 2020, 69(24): 244204. doi: 10.7498/aps.69.20200805
    [20] 陈炜, 郭媛, 敬世伟. 基于深度学习压缩感知与复合混沌系统的通用图像加密算法. 物理学报, 2020, 69(24): 240502. doi: 10.7498/aps.69.20201019
计量
  • 文章访问数:  5645
  • PDF下载量:  128
  • 被引次数: 0
出版历程
  • 收稿日期:  2020-09-09
  • 修回日期:  2020-11-22
  • 上网日期:  2021-03-24
  • 刊出日期:  2021-04-05

/

返回文章
返回