搜索

x

留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

基于数字微镜器件的快速超分辨晶格结构光照明显微研究

杨浩智 聂梦娇 马光鹏 曹慧群 林丹樱 屈军乐 于斌

引用本文:
Citation:

基于数字微镜器件的快速超分辨晶格结构光照明显微研究

杨浩智, 聂梦娇, 马光鹏, 曹慧群, 林丹樱, 屈军乐, 于斌

Digital micromirror device-based fast super-resolution lattice structured light illumination microscopy

Yang Hao-Zhi, Nie Meng-Jiao, Ma Guang-Peng, Cao Hui-Qun, Lin Dan-Ying, Qu Jun-Le, Yu Bin
PDF
HTML
导出引用
  • 超分辨结构光照明显微成像技术(super-resolution structured illumination microscopy, SR-SIM)具有时间分辨率高、光漂白和光毒性低和对荧光探针的要求少等优点, 适用于活细胞的长时程超分辨成像. 采用二维晶格结构光作为照明光, 可以实现更快的成像速度和更低的光毒性, 但同时也增加了系统的复杂性. 为了解决此问题, 本文提出了一种基于数字微镜器件的快速超分辨晶格结构光照明显微成像方法(digital micromirror device-based lattice SIM , DMD-Lattice-SIM), 通过同步分时触发DMD和sCMOS相机的方式实现二维正交晶格结构光的产生, 且只需要采集5幅相移原始图像即可重构出超分辨图像, 相比于传统SR-SIM需要9幅相移原始图像的方法, 图像采集时间减少了约44.4%. 同时, 在基于空域和频域联合的SIM重构算法(joint space and frequency reconstruction method-SIM, JSFR-SIM)的基础上, 本文还发展了用于Lattice-SIM的JSFR超分辨图像重构方法(Lattice-JSFR-SIM), 先在频域对原始图像进行预滤波处理; 然后, 在空域对滤波后的图像进行超分辨重构处理. 与传统频域图像重构处理对比, 该方法在512 ×512 像素数的成像视场下重构时间减少了约55.6%, 对于实现活细胞实时超分辨成像具有重要意义和应用价值.
    Super-resolution structured illumination microscopy (SR-SIM) offers numerous advantages such as high temporal resolution, low photobleaching and phototoxicity, and no special requirements for fluorescent probes. It is particularly suitable for long-term SR imaging of living cells. By using two-dimensional lattice structured light serving as illumination, SR-SIM can achieve faster imaging speed and reduce phototoxicity, however, it is accompanied with system complexity increasing. To address this problem, in this work, we propose a fast SR lattice structured illumination microscopy imaging method based on a digital micromirror device (DMD), called DMD-Lattice-SIM. This method utilizes a DMD and synchronous time-sharing triggering with sCMOS to generate two-dimensional orthogonal lattice structured light. The proposed method only requires the collection of five phase-shifted raw images for SR image reconstruction, reducing the acquisition time by approximately 44.4% compared with the traditional SR-SIM method that requires nine phase-shifted raw images. In this work, we also introduce a rapid SR image reconstruction method called Lattice-JSFR-SIM, which combines the advantages of joint space and frequency reconstruction (JSFR)-SIM and Lattice-SIM. The raw images are pre-filtered in the frequency domain and then undergo SR reconstruction in the spatial domain. This approach reduces reconstruction time by approximately 55.6% compared with traditional frequency domain image reconstruction processing, within an imaging field of view of 512 pixels×512 pixels. The feasibility of the proposed method is demonstrated through experiments on cell microtubules and the observation of mitochondrial division and fusion in living cells. The findings presented in this paper hold great significance and application value for enabling real-time SR imaging of living cells.
      通信作者: 于斌, yubin@szu.edu.cn
    • 基金项目: 国家重点研发计划(批准号: 2022YFF0712500)、国家自然科学基金(批准号: 62175166, 61975131, 62235007, 62275165, 62127819)、深圳市科技计划(批准号: JCYJ20200109105411133, JCYJ20220818100202005)和深圳市光子学和生物光子学重点实验室(批准号: ZDSYS20210623092006020)资助的课题.
      Corresponding author: Yu Bin, yubin@szu.edu.cn
    • Funds: Project supported by the National Key Research and Development Program of China (Grant No. 2022YFF0712500), the National Natural Science Foundation of China (Grant Nos. 62175166, 61975131, 62235007, 62275165, 62127819), the Science and Technology Planning Project of Shenzhen, China (Grant Nos. JCYJ20200109105411133, JCYJ20220818100202005), and the Shenzhen Key Laboratory of Photonics and Biophotonics, China (Grant No. ZDSYS20210623092006020).
    [1]

    Gustafsson M G L 2000 J. Microsc.-Oxford 198 82Google Scholar

    [2]

    Chen X, Zhong S Y, Hou Y W, Cao R J, Wang W Y, Li D, Dai Q H, Kim D, Xi P 2023 Light-Sci. Appl. 12 172Google Scholar

    [3]

    Chang B J, Chou L J, Chang Y C, Chiang S Y 2009 Opt. Express 17 14710Google Scholar

    [4]

    Dan D, Lei M, Yao B L, Wang W, Winterhalder M, Zumbusch A, Qi Y J, Xia L, Yan S H, Yang Y L, Gao P, Ye T, Zhao, W 2013 Sci. Rep. 3 1116Google Scholar

    [5]

    Li M Q, Li Y N, Liu W H, Lal A, Jiang S, Jin D Y, Yang H P, Wang S, Zhanghao K, Xi P 2020 Appl. Phys. Lett. 116 233702Google Scholar

    [6]

    Li X Y, Xie S Y, Liu W J, Jin L H, Xu Y K, Zhang L H, Hao X, Han Y B, Kuang C F, Liu X 2021 Opt. Express 29 43917Google Scholar

    [7]

    Mudry E, Belkebir K, Girard J, Savatier J, Le Moal E, Nicoletti C, Allain M, Sentenac A 2012 Nat. Photonics 6 312Google Scholar

    [8]

    Heintzmann R 2003 Micron 34 283Google Scholar

    [9]

    Siebenmorgen J, Novikau Y, Wolleschensky R, Weisshart K, Kleppe I 2018 Introducing Lattice SIM for ZEISS Elyra 7 (Germany: Carl Zeiss Microscopy GmbH) pp1–7

    [10]

    Zheng J J, Fang X, Wen K, Li J Y, Ma Y, Liu M, An S, Li J L, Zalevsky Z, Gao P 2022 Opt. Express 30 27951Google Scholar

    [11]

    Huang X S, Fan J C, Li L J, Liu H S, Wu R L, Wu Y, Wei L S, Mao H, Lal A, Xi P, Tang L Q, Zhang Y F, Liu Y M, Tan S, Chen L Y 2018 Nat. Biotechnol. 36 451Google Scholar

    [12]

    Zhao W S, Zhao S Q, Li L J, Huang X S, Xing S J, Zhang Y L, Qiu G H, Han Z Q, Shang Y X, Sun D E, Shan C Y, Wu R L, Gu L S, Zhang S W, Chen R W, Xiao J, Mo Y Q, Wang J Y, Ji W, Chen X, Ding B Q, Liu Y M, Mao H, Song B L, Tan J B, Liu J, Li H Y, Chen L Y 2022 Nat. Biotechnol. 40 606Google Scholar

    [13]

    Wen G, Li S M, Wang L B, Chen X H, Sun Z L, Liang Y, Jin X, Xing Y F, Jiu Y M, Tang Y G, Li H 2021 Light-Sci. Appl. 10 70Google Scholar

    [14]

    Tu S J, Liu Q L, Liu X, Liu W J, Zhang Z M, Luo T J, Kuang C F, Liu X, Hao X 2020 Opt. Lett. 45 1567Google Scholar

    [15]

    Dan D, Wang Z J, Zhou X, Lei M, Zhao T Y, Qian J, Yu X H, Yan S H, Min J W, Bianco P, Yao B L 2021 IEEE Photonics J. 13 3900411Google Scholar

    [16]

    Wang Z J, Zhao T Y, Hao H W, Cai Y N, Feng K, Yun X, Liang Y S, Wang S W, Sun Y J, Bianco P R, Oh K, Lei M 2022 Adv. Photonics 4 026003Google Scholar

    [17]

    Wang Z J, Zhao T Y, Cai Y A, Zhang J X, Hao H W, Liang Y S, Wang S W, Sun Y J, Chen T S, Bianco P R, Oh K, Lei M 2023 The Innovation 4 100425Google Scholar

    [18]

    Wen G, Li S M, Liang Y, Wang L B, Zhang J, Chen X H, Jin X, Chen C, Tang Y G, Li H 2023 PhotoniX 4 19Google Scholar

    [19]

    Müller M, Mönkemöller V, Hennig S, Hübner W, Huser T 2016 Nat. Commun. 7 10980Google Scholar

    [20]

    Zhou L L, Yu B, Huang L L, Cao H Q, Lin D Y, Jing Y Y, Wali F, Qu J L 2022 ACS Appl. Nano Mater. 5 18742Google Scholar

    [21]

    Descloux A, Grussmayer K S, Radenovic A 2019 Nat. Methods 16 918Google Scholar

  • 图 1  DMD-Lattice-SIM系统光路示意图

    Fig. 1.  Schematic diagram of DMD-Lattice-SIM system.

    图 2  产生晶格结构光照明的原理示意图 (a) 数据采集卡的控制信号; (b) DMD加载第一张模板时相机采集的模拟数据; (c) DMD加载第二张模板时相机采集的模拟数据; (d) 相机在一次曝光时间内采集的模拟数据; (e)—(g) 图(b)—(d)的傅里叶频谱

    Fig. 2.  Schematic diagram of the principle of generating lattice structured illumination: (a) Control signal of the data acquisition board; (b) analog data collected by the camera when the DMD loads the first template; (c) analog data collected by the camera when the DMD loads the second template; (d) analog data collected by the camera within one exposure time; (e)–(g) frequency spectra of panel (b)–(d).

    图 3  Lattice-JSFR-SIM重构过程

    Fig. 3.  Lattice-JSFR-SIM reconstruction process.

    图 4  模拟数据重构结果 (a) 真实图像; (b) 宽场图像; (c) fairSIM超分辨重构图像; (d) Lattice-SIM超分辨重构图像; (e) Lattice-JSFR-SIM超分辨重构图像; (f) 图(b)—(e)中曲线的切面强度分布

    Fig. 4.  Reconstructed results of the simulated data: (a) Ground truth; (b) wide-field image; (c) fairSIM super-resolution reconstructed image; (d) Lattice-SIM super-resolution reconstructed image; (e) Lattice-JSFR-SIM super-resolution reconstructed image; (f) intensity distribution of the profile in panel (b)–(e).

    图 5  固定细胞微管实验结果 (a) 宽场图像; (b) 超分辨图像; (c) 图(a)中所选区域的放大; (d) 图(b)中所选区域的放大图; (e) 图(a)的去相关分辨率估计; (f) 图(b)的去相关分辨率估计; (g) 图(c)和图(d)中划线位置的切面强度分布

    Fig. 5.  Experimental results of fixed cell microtubules: (a) Wide-field image; (b) super-resolution image; (c) magnification of the selected area in panel (a); (d) magnification of the selected area in panel (b); (e) corresponding decorrelation analysis of panel (a), C.c., cross-correlation; (f) corresponding decorrelation analysis of panel (b); (g) intensity distribution of the profile at the line position in panel (c) and (d).

    图 6  BSC-1活细胞线粒体动态实验结果 (a) 宽场图像; (b) 超分辨图像; (c) 图(a)的去相关分辨率估计; (d) 图(b)的去相关分辨率估计; (e) 图(b)中所选区域放大图, 时间间隔为0.5 s; (f) 图(b)中所选区域放大图, 时间间隔为0.05 s

    Fig. 6.  Experimental results of BSC-1 live cell mitochondria dynamics: (a) Wide-field image; (b) super-resolution image; (c) corresponding decorrelation analysis of panel (a); (d) corresponding decorrelation analysis of panel (b); (e) magnification of the selected area in panel (b) with a time interval of 0.5 s; (f) magnification of the selected area in panel (b) with a time interval of 0.05 s.

    表 1  不同照明图案的SIM对比

    Table 1.  Comparison of SIM with different illumination patterns.

    名称 照明图案 特点 优缺点
    blind-SIM 散斑 使用散斑照明代替传统条纹照明 不需要求解照明图案参数, 简化了图像重构过程; 但需要采集较多的原始图像进行迭代重建, 成像速度慢
    Lattice SIM for ZEISS Elyra 7 正方晶格、
    六边形晶格
    物理光栅产生晶格, 振镜产生相移, 并且结合解卷积算法重构 成像速度和成像对比度高, 实现了60 nm的空间分辨率
    Lattice-SIM 正方晶格 物理光栅产生晶格, SLM产生相移 成像视场大, 但系统复杂, 速度慢
    下载: 导出CSV

    表 2  不同重构算法的SIM对比

    Table 2.  Comparison of reconstruction algorithms.

    名称 类别 特点 优缺点
    Hessian-SIM 迭代 利用生物结构的连续性进行重建 减少了重构伪影, 光毒性低、成像速度快、
    分辨率高, 但参数设置敏感
    Sparse-SIM 迭代 将生物结构的稀疏性加入到
    Hessian-SIM中进行重建
    分辨率高, 但需要人为控制重构参数设置
    HiFi-SIM 频域 利用点扩散函数改造方法优化频谱 图像伪影少、光学层切能力高、保真度高、参数设置不敏感
    SP-SIM 空域 无需估计光照模式的频率和相位 重构速度比传统算法快约5.4倍, 但在重构过程中
    丢失了低频分量, 导致图像对比度低
    SDR-SIM 空域 利用数学中函数的级数展开的概念 重构速度比传统算法快约7倍, 但没有考虑离焦背景的问题
    JSFR-SIM 空频域混合 首先在频域预滤波, 然后进行空域重构 图像重建速度高、背景荧光和周期性计算伪影低,
    相比Wiener-SIM算法快近两个数量级
    direct-SIM 空频域混合 首先进行空域重构, 然后进行频域频谱优化 无需估计结构光条纹参数、分辨率高、伪影低、
    样本要求低, 但对条纹对比度和相移精确度要求较高
    下载: 导出CSV

    表 3  算法运行速度对比

    Table 3.  Comparison of algorithm running speed.

    Input image size/
    (pixels×pixels)
    Output image size/
    (pixels×pixels)
    Acquisition
    time/ms
    Lattice-SIM/ms Lattice-JSFR-SIM/ms
    CPU CPU GPU
    256×256 512×512 6.4 137.9±6.3 44.9±2.4 4.3±3.7
    512×512 1024×1024 12.7 751.0±26.1 333.6±14.0 17.6±4.2
    1024×1024 2048×2048 31.3 3007.2±58.8 1749.8±22.9 75.8±6.3
    下载: 导出CSV
  • [1]

    Gustafsson M G L 2000 J. Microsc.-Oxford 198 82Google Scholar

    [2]

    Chen X, Zhong S Y, Hou Y W, Cao R J, Wang W Y, Li D, Dai Q H, Kim D, Xi P 2023 Light-Sci. Appl. 12 172Google Scholar

    [3]

    Chang B J, Chou L J, Chang Y C, Chiang S Y 2009 Opt. Express 17 14710Google Scholar

    [4]

    Dan D, Lei M, Yao B L, Wang W, Winterhalder M, Zumbusch A, Qi Y J, Xia L, Yan S H, Yang Y L, Gao P, Ye T, Zhao, W 2013 Sci. Rep. 3 1116Google Scholar

    [5]

    Li M Q, Li Y N, Liu W H, Lal A, Jiang S, Jin D Y, Yang H P, Wang S, Zhanghao K, Xi P 2020 Appl. Phys. Lett. 116 233702Google Scholar

    [6]

    Li X Y, Xie S Y, Liu W J, Jin L H, Xu Y K, Zhang L H, Hao X, Han Y B, Kuang C F, Liu X 2021 Opt. Express 29 43917Google Scholar

    [7]

    Mudry E, Belkebir K, Girard J, Savatier J, Le Moal E, Nicoletti C, Allain M, Sentenac A 2012 Nat. Photonics 6 312Google Scholar

    [8]

    Heintzmann R 2003 Micron 34 283Google Scholar

    [9]

    Siebenmorgen J, Novikau Y, Wolleschensky R, Weisshart K, Kleppe I 2018 Introducing Lattice SIM for ZEISS Elyra 7 (Germany: Carl Zeiss Microscopy GmbH) pp1–7

    [10]

    Zheng J J, Fang X, Wen K, Li J Y, Ma Y, Liu M, An S, Li J L, Zalevsky Z, Gao P 2022 Opt. Express 30 27951Google Scholar

    [11]

    Huang X S, Fan J C, Li L J, Liu H S, Wu R L, Wu Y, Wei L S, Mao H, Lal A, Xi P, Tang L Q, Zhang Y F, Liu Y M, Tan S, Chen L Y 2018 Nat. Biotechnol. 36 451Google Scholar

    [12]

    Zhao W S, Zhao S Q, Li L J, Huang X S, Xing S J, Zhang Y L, Qiu G H, Han Z Q, Shang Y X, Sun D E, Shan C Y, Wu R L, Gu L S, Zhang S W, Chen R W, Xiao J, Mo Y Q, Wang J Y, Ji W, Chen X, Ding B Q, Liu Y M, Mao H, Song B L, Tan J B, Liu J, Li H Y, Chen L Y 2022 Nat. Biotechnol. 40 606Google Scholar

    [13]

    Wen G, Li S M, Wang L B, Chen X H, Sun Z L, Liang Y, Jin X, Xing Y F, Jiu Y M, Tang Y G, Li H 2021 Light-Sci. Appl. 10 70Google Scholar

    [14]

    Tu S J, Liu Q L, Liu X, Liu W J, Zhang Z M, Luo T J, Kuang C F, Liu X, Hao X 2020 Opt. Lett. 45 1567Google Scholar

    [15]

    Dan D, Wang Z J, Zhou X, Lei M, Zhao T Y, Qian J, Yu X H, Yan S H, Min J W, Bianco P, Yao B L 2021 IEEE Photonics J. 13 3900411Google Scholar

    [16]

    Wang Z J, Zhao T Y, Hao H W, Cai Y N, Feng K, Yun X, Liang Y S, Wang S W, Sun Y J, Bianco P R, Oh K, Lei M 2022 Adv. Photonics 4 026003Google Scholar

    [17]

    Wang Z J, Zhao T Y, Cai Y A, Zhang J X, Hao H W, Liang Y S, Wang S W, Sun Y J, Chen T S, Bianco P R, Oh K, Lei M 2023 The Innovation 4 100425Google Scholar

    [18]

    Wen G, Li S M, Liang Y, Wang L B, Zhang J, Chen X H, Jin X, Chen C, Tang Y G, Li H 2023 PhotoniX 4 19Google Scholar

    [19]

    Müller M, Mönkemöller V, Hennig S, Hübner W, Huser T 2016 Nat. Commun. 7 10980Google Scholar

    [20]

    Zhou L L, Yu B, Huang L L, Cao H Q, Lin D Y, Jing Y Y, Wali F, Qu J L 2022 ACS Appl. Nano Mater. 5 18742Google Scholar

    [21]

    Descloux A, Grussmayer K S, Radenovic A 2019 Nat. Methods 16 918Google Scholar

  • [1] 罗泽伟, 武戈, 陈挚, 邓驰楠, 万蓉, 杨涛, 庄正飞, 陈同生. 双通道结构光照明超分辨定量荧光共振能量转移成像系统. 物理学报, 2023, 72(20): 208701. doi: 10.7498/aps.72.20230853
    [2] 凌进中, 郭金坤, 王昱程, 刘鑫, 王晓蕊. 基于倏逝波照明的空间移频超分辨成像技术研究. 物理学报, 2023, 72(22): 224202. doi: 10.7498/aps.72.20230934
    [3] 高兆琳, 刘瑞桦, 温凯, 马英, 李建郎, 郜鹏. 结构光照明相位/荧光双模式显微技术. 物理学报, 2022, 71(24): 244203. doi: 10.7498/aps.71.20221518
    [4] 葛阳阳, 何灼奋, 黄黎琳, 林丹樱, 曹慧群, 屈军乐, 于斌. 平场复用多焦点结构光照明超分辨显微成像. 物理学报, 2022, 71(4): 048704. doi: 10.7498/aps.71.20211712
    [5] 葛阳阳, 于斌. 平场复用多焦点结构光照明超分辨显微成像研究. 物理学报, 2021, (): . doi: 10.7498/aps.70.20211712
    [6] 刘康, 何韬, 刘涛, 李国卿, 田博, 王佳怡, 杨树明. 激光照明条件对超振荡平面透镜聚焦性能的影响. 物理学报, 2020, 69(18): 184215. doi: 10.7498/aps.69.20200577
    [7] 千佳, 党诗沛, 周兴, 但旦, 汪召军, 赵天宇, 梁言生, 姚保利, 雷铭. 基于希尔伯特变换的结构光照明快速三维彩色显微成像方法. 物理学报, 2020, 69(12): 128701. doi: 10.7498/aps.69.20200352
    [8] 张佳, SamantaSoham, 王佳林, 王璐玮, 杨志刚, 严伟, 屈军乐. 一种用于线粒体受激辐射损耗超分辨成像的新型探针. 物理学报, 2020, 69(16): 168702. doi: 10.7498/aps.69.20200171
    [9] 肖晓, 杜舒曼, 赵富, 王晶, 刘军, 李儒新. 基于赝热光照明的单发光学散斑成像. 物理学报, 2019, 68(3): 034201. doi: 10.7498/aps.68.20181723
    [10] 闫博, 陈力, 陈爽, 李猛, 殷一民, 周江宁. 结构光照明技术在二维激光诱导荧光成像去杂散光中的应用. 物理学报, 2019, 68(21): 218701. doi: 10.7498/aps.68.20190977
    [11] 田源, 葛浩, 卢明辉, 陈延峰. 声学超构材料及其物理效应的研究进展. 物理学报, 2019, 68(19): 194301. doi: 10.7498/aps.68.20190850
    [12] 刘雄波, 林丹樱, 吴茜茜, 严伟, 罗腾, 杨志刚, 屈军乐. 荧光寿命显微成像技术及应用的最新研究进展. 物理学报, 2018, 67(17): 178701. doi: 10.7498/aps.67.20180320
    [13] 赵天宇, 周兴, 但旦, 千佳, 汪召军, 雷铭, 姚保利. 结构光照明显微中的偏振控制. 物理学报, 2017, 66(14): 148704. doi: 10.7498/aps.66.148704
    [14] 林丹樱, 屈军乐. 超分辨成像及超分辨关联显微技术研究进展. 物理学报, 2017, 66(14): 148703. doi: 10.7498/aps.66.148703
    [15] 张崇磊, 辛自强, 闵长俊, 袁小聪. 表面等离激元结构光照明显微成像技术研究进展. 物理学报, 2017, 66(14): 148701. doi: 10.7498/aps.66.148701
    [16] 刘鸿吉, 刘双龙, 牛憨笨, 陈丹妮, 刘伟. 基于环形抽运光的红外超分辨显微成像方法. 物理学报, 2016, 65(23): 233601. doi: 10.7498/aps.65.233601
    [17] 赵应春, 张秀英, 袁操今, 聂守平, 朱竹青, 王林, 李杨, 贡丽萍, 冯少彤. 基于涡旋光照明的暗场数字全息显微方法研究. 物理学报, 2014, 63(22): 224202. doi: 10.7498/aps.63.224202
    [18] 李恒, 于斌, 陈丹妮, 牛憨笨. 高效双螺旋点扩展函数相位片的设计与实验研究. 物理学报, 2013, 62(12): 124201. doi: 10.7498/aps.62.124201
    [19] 支绍韬, 章海军, 张冬仙. 基于大数值孔径环形光锥照明的超分辨光学显微成像方法研究. 物理学报, 2012, 61(2): 024207. doi: 10.7498/aps.61.024207
    [20] 梁高峰, 赵青, 陈欣, 王长涛, 赵泽宇, 罗先刚. 基于多层膜结构的亚波长光栅研究. 物理学报, 2012, 61(10): 104203. doi: 10.7498/aps.61.104203
计量
  • 文章访问数:  2764
  • PDF下载量:  103
  • 被引次数: 0
出版历程
  • 收稿日期:  2024-01-31
  • 修回日期:  2024-02-27
  • 上网日期:  2024-03-12
  • 刊出日期:  2024-05-05

/

返回文章
返回