Digital micromirror device-based fast super-resolution lattice structured light illumination microscopy

Yang Hao-Zhi; Nie Meng-Jiao; Ma Guang-Peng; Cao Hui-Qun; Lin Dan-Ying; Qu Jun-Le; Yu Bin

doi:10.7498/aps.73.20240216

Abstract

Super-resolution structured illumination microscopy (SR-SIM) offers numerous advantages such as high temporal resolution, low photobleaching and phototoxicity, and no special requirements for fluorescent probes. It is particularly suitable for long-term SR imaging of living cells. By using two-dimensional lattice structured light serving as illumination, SR-SIM can achieve faster imaging speed and reduce phototoxicity, however, it is accompanied with system complexity increasing. To address this problem, in this work, we propose a fast SR lattice structured illumination microscopy imaging method based on a digital micromirror device (DMD), called DMD-Lattice-SIM. This method utilizes a DMD and synchronous time-sharing triggering with sCMOS to generate two-dimensional orthogonal lattice structured light. The proposed method only requires the collection of five phase-shifted raw images for SR image reconstruction, reducing the acquisition time by approximately 44.4% compared with the traditional SR-SIM method that requires nine phase-shifted raw images. In this work, we also introduce a rapid SR image reconstruction method called Lattice-JSFR-SIM, which combines the advantages of joint space and frequency reconstruction (JSFR)-SIM and Lattice-SIM. The raw images are pre-filtered in the frequency domain and then undergo SR reconstruction in the spatial domain. This approach reduces reconstruction time by approximately 55.6% compared with traditional frequency domain image reconstruction processing, within an imaging field of view of 512 pixels×512 pixels. The feasibility of the proposed method is demonstrated through experiments on cell microtubules and the observation of mitochondrial division and fusion in living cells. The findings presented in this paper hold great significance and application value for enabling real-time SR imaging of living cells.

Keywords:

Authors and contacts

1.
Key Laboratory of Optoelectronic Devices and Systems of Ministry of Education and Guangdong Province, College of Physics and Optoelectronic Engineering, Shenzhen University, Shenzhen 518060, China
2.
College of Chemistry and Environmental Engineering, Shenzhen University, Shenzhen 518060, China

Corresponding author: Yu Bin, yubin@szu.edu.cn

Funds: Project supported by the National Key Research and Development Program of China (Grant No. 2022YFF0712500), the National Natural Science Foundation of China (Grant Nos. 62175166, 61975131, 62235007, 62275165, 62127819), the Science and Technology Planning Project of Shenzhen, China (Grant Nos. JCYJ20200109105411133, JCYJ20220818100202005), and the Shenzhen Key Laboratory of Photonics and Biophotonics, China (Grant No. ZDSYS20210623092006020).

FullText HTML : translate this paragraph

References

[1]	Gustafsson M G L 2000 J. Microsc.-Oxford 198 82 Google Scholar
[2]	Chen X, Zhong S Y, Hou Y W, Cao R J, Wang W Y, Li D, Dai Q H, Kim D, Xi P 2023 Light-Sci. Appl. 12 172 Google Scholar
[3]	Chang B J, Chou L J, Chang Y C, Chiang S Y 2009 Opt. Express 17 14710 Google Scholar
[4]	Dan D, Lei M, Yao B L, Wang W, Winterhalder M, Zumbusch A, Qi Y J, Xia L, Yan S H, Yang Y L, Gao P, Ye T, Zhao, W 2013 Sci. Rep. 3 1116 Google Scholar
[5]	Li M Q, Li Y N, Liu W H, Lal A, Jiang S, Jin D Y, Yang H P, Wang S, Zhanghao K, Xi P 2020 Appl. Phys. Lett. 116 233702 Google Scholar
[6]	Li X Y, Xie S Y, Liu W J, Jin L H, Xu Y K, Zhang L H, Hao X, Han Y B, Kuang C F, Liu X 2021 Opt. Express 29 43917 Google Scholar
[7]	Mudry E, Belkebir K, Girard J, Savatier J, Le Moal E, Nicoletti C, Allain M, Sentenac A 2012 Nat. Photonics 6 312 Google Scholar
[8]	Heintzmann R 2003 Micron 34 283 Google Scholar
[9]	Siebenmorgen J, Novikau Y, Wolleschensky R, Weisshart K, Kleppe I 2018 Introducing Lattice SIM for ZEISS Elyra 7 (Germany: Carl Zeiss Microscopy GmbH) pp1–7
[10]	Zheng J J, Fang X, Wen K, Li J Y, Ma Y, Liu M, An S, Li J L, Zalevsky Z, Gao P 2022 Opt. Express 30 27951 Google Scholar
[11]	Huang X S, Fan J C, Li L J, Liu H S, Wu R L, Wu Y, Wei L S, Mao H, Lal A, Xi P, Tang L Q, Zhang Y F, Liu Y M, Tan S, Chen L Y 2018 Nat. Biotechnol. 36 451 Google Scholar
[12]	Zhao W S, Zhao S Q, Li L J, Huang X S, Xing S J, Zhang Y L, Qiu G H, Han Z Q, Shang Y X, Sun D E, Shan C Y, Wu R L, Gu L S, Zhang S W, Chen R W, Xiao J, Mo Y Q, Wang J Y, Ji W, Chen X, Ding B Q, Liu Y M, Mao H, Song B L, Tan J B, Liu J, Li H Y, Chen L Y 2022 Nat. Biotechnol. 40 606 Google Scholar
[13]	Wen G, Li S M, Wang L B, Chen X H, Sun Z L, Liang Y, Jin X, Xing Y F, Jiu Y M, Tang Y G, Li H 2021 Light-Sci. Appl. 10 70 Google Scholar
[14]	Tu S J, Liu Q L, Liu X, Liu W J, Zhang Z M, Luo T J, Kuang C F, Liu X, Hao X 2020 Opt. Lett. 45 1567 Google Scholar
[15]	Dan D, Wang Z J, Zhou X, Lei M, Zhao T Y, Qian J, Yu X H, Yan S H, Min J W, Bianco P, Yao B L 2021 IEEE Photonics J. 13 3900411 Google Scholar
[16]	Wang Z J, Zhao T Y, Hao H W, Cai Y N, Feng K, Yun X, Liang Y S, Wang S W, Sun Y J, Bianco P R, Oh K, Lei M 2022 Adv. Photonics 4 026003 Google Scholar
[17]	Wang Z J, Zhao T Y, Cai Y A, Zhang J X, Hao H W, Liang Y S, Wang S W, Sun Y J, Chen T S, Bianco P R, Oh K, Lei M 2023 The Innovation 4 100425 Google Scholar
[18]	Wen G, Li S M, Liang Y, Wang L B, Zhang J, Chen X H, Jin X, Chen C, Tang Y G, Li H 2023 PhotoniX 4 19 Google Scholar
[19]	Müller M, Mönkemöller V, Hennig S, Hübner W, Huser T 2016 Nat. Commun. 7 10980 Google Scholar
[20]	Zhou L L, Yu B, Huang L L, Cao H Q, Lin D Y, Jing Y Y, Wali F, Qu J L 2022 ACS Appl. Nano Mater. 5 18742 Google Scholar
[21]	Descloux A, Grussmayer K S, Radenovic A 2019 Nat. Methods 16 918 Google Scholar

Cited By

图 1 DMD-Lattice-SIM系统光路示意图

Figure 1. Schematic diagram of DMD-Lattice-SIM system.

DownLoad: Full-Size Img PowerPoint

图 2 产生晶格结构光照明的原理示意图　(a) 数据采集卡的控制信号; (b) DMD加载第一张模板时相机采集的模拟数据; (c) DMD加载第二张模板时相机采集的模拟数据; (d) 相机在一次曝光时间内采集的模拟数据; (e)—(g) 图(b)—(d)的傅里叶频谱

Figure 2. Schematic diagram of the principle of generating lattice structured illumination: (a) Control signal of the data acquisition board; (b) analog data collected by the camera when the DMD loads the first template; (c) analog data collected by the camera when the DMD loads the second template; (d) analog data collected by the camera within one exposure time; (e)–(g) frequency spectra of panel (b)–(d).

DownLoad: Full-Size Img PowerPoint

图 3 Lattice-JSFR-SIM重构过程

Figure 3. Lattice-JSFR-SIM reconstruction process.

DownLoad: Full-Size Img PowerPoint

图 4 模拟数据重构结果　(a) 真实图像; (b) 宽场图像; (c) fairSIM超分辨重构图像; (d) Lattice-SIM超分辨重构图像; (e) Lattice-JSFR-SIM超分辨重构图像; (f) 图(b)—(e)中曲线的切面强度分布

Figure 4. Reconstructed results of the simulated data: (a) Ground truth; (b) wide-field image; (c) fairSIM super-resolution reconstructed image; (d) Lattice-SIM super-resolution reconstructed image; (e) Lattice-JSFR-SIM super-resolution reconstructed image; (f) intensity distribution of the profile in panel (b)–(e).

DownLoad: Full-Size Img PowerPoint

图 5 固定细胞微管实验结果　(a) 宽场图像; (b) 超分辨图像; (c) 图(a)中所选区域的放大; (d) 图(b)中所选区域的放大图; (e) 图(a)的去相关分辨率估计; (f) 图(b)的去相关分辨率估计; (g) 图(c)和图(d)中划线位置的切面强度分布

Figure 5. Experimental results of fixed cell microtubules: (a) Wide-field image; (b) super-resolution image; (c) magnification of the selected area in panel (a); (d) magnification of the selected area in panel (b); (e) corresponding decorrelation analysis of panel (a), C.c., cross-correlation; (f) corresponding decorrelation analysis of panel (b); (g) intensity distribution of the profile at the line position in panel (c) and (d).

DownLoad: Full-Size Img PowerPoint

图 6 BSC-1活细胞线粒体动态实验结果　(a) 宽场图像; (b) 超分辨图像; (c) 图(a)的去相关分辨率估计; (d) 图(b)的去相关分辨率估计; (e) 图(b)中所选区域放大图, 时间间隔为0.5 s; (f) 图(b)中所选区域放大图, 时间间隔为0.05 s

Figure 6. Experimental results of BSC-1 live cell mitochondria dynamics: (a) Wide-field image; (b) super-resolution image; (c) corresponding decorrelation analysis of panel (a); (d) corresponding decorrelation analysis of panel (b); (e) magnification of the selected area in panel (b) with a time interval of 0.5 s; (f) magnification of the selected area in panel (b) with a time interval of 0.05 s.

DownLoad: Full-Size Img PowerPoint

表 1 不同照明图案的SIM对比

Table 1. Comparison of SIM with different illumination patterns.

名称	照明图案	特点	优缺点
blind-SIM	散斑	使用散斑照明代替传统条纹照明	不需要求解照明图案参数, 简化了图像重构过程; 但需要采集较多的原始图像进行迭代重建, 成像速度慢
Lattice SIM for ZEISS Elyra 7	正方晶格、六边形晶格	物理光栅产生晶格, 振镜产生相移, 并且结合解卷积算法重构	成像速度和成像对比度高, 实现了60 nm的空间分辨率
Lattice-SIM	正方晶格	物理光栅产生晶格, SLM产生相移	成像视场大, 但系统复杂, 速度慢

DownLoad: CSV

表 2 不同重构算法的SIM对比

Table 2. Comparison of reconstruction algorithms.

名称	类别	特点	优缺点
Hessian-SIM	迭代	利用生物结构的连续性进行重建	减少了重构伪影, 光毒性低、成像速度快、分辨率高, 但参数设置敏感
Sparse-SIM	迭代	将生物结构的稀疏性加入到 Hessian-SIM中进行重建	分辨率高, 但需要人为控制重构参数设置
HiFi-SIM	频域	利用点扩散函数改造方法优化频谱	图像伪影少、光学层切能力高、保真度高、参数设置不敏感
SP-SIM	空域	无需估计光照模式的频率和相位	重构速度比传统算法快约5.4倍, 但在重构过程中丢失了低频分量, 导致图像对比度低
SDR-SIM	空域	利用数学中函数的级数展开的概念	重构速度比传统算法快约7倍, 但没有考虑离焦背景的问题
JSFR-SIM	空频域混合	首先在频域预滤波, 然后进行空域重构	图像重建速度高、背景荧光和周期性计算伪影低, 相比Wiener-SIM算法快近两个数量级
direct-SIM	空频域混合	首先进行空域重构, 然后进行频域频谱优化	无需估计结构光条纹参数、分辨率高、伪影低、样本要求低, 但对条纹对比度和相移精确度要求较高

DownLoad: CSV

表 3 算法运行速度对比

Table 3. Comparison of algorithm running speed.

Input image size/ (pixels×pixels)	Output image size/ (pixels×pixels)	Acquisition time/ms	Lattice-SIM/ms	Lattice-JSFR-SIM/ms
Input image size/ (pixels×pixels)	Output image size/ (pixels×pixels)	Acquisition time/ms	CPU	CPU	GPU
256×256	512×512	6.4	137.9±6.3	44.9±2.4	4.3±3.7
512×512	1024×1024	12.7	751.0±26.1	333.6±14.0	17.6±4.2
1024×1024	2048×2048	31.3	3007.2±58.8	1749.8±22.9	75.8±6.3

DownLoad: CSV

[1]	Gustafsson M G L 2000 J. Microsc.-Oxford 198 82 Google Scholar
[2]	Chen X, Zhong S Y, Hou Y W, Cao R J, Wang W Y, Li D, Dai Q H, Kim D, Xi P 2023 Light-Sci. Appl. 12 172 Google Scholar
[3]	Chang B J, Chou L J, Chang Y C, Chiang S Y 2009 Opt. Express 17 14710 Google Scholar
[4]	Dan D, Lei M, Yao B L, Wang W, Winterhalder M, Zumbusch A, Qi Y J, Xia L, Yan S H, Yang Y L, Gao P, Ye T, Zhao, W 2013 Sci. Rep. 3 1116 Google Scholar
[5]	Li M Q, Li Y N, Liu W H, Lal A, Jiang S, Jin D Y, Yang H P, Wang S, Zhanghao K, Xi P 2020 Appl. Phys. Lett. 116 233702 Google Scholar
[6]	Li X Y, Xie S Y, Liu W J, Jin L H, Xu Y K, Zhang L H, Hao X, Han Y B, Kuang C F, Liu X 2021 Opt. Express 29 43917 Google Scholar
[7]	Mudry E, Belkebir K, Girard J, Savatier J, Le Moal E, Nicoletti C, Allain M, Sentenac A 2012 Nat. Photonics 6 312 Google Scholar
[8]	Heintzmann R 2003 Micron 34 283 Google Scholar
[9]	Siebenmorgen J, Novikau Y, Wolleschensky R, Weisshart K, Kleppe I 2018 Introducing Lattice SIM for ZEISS Elyra 7 (Germany: Carl Zeiss Microscopy GmbH) pp1–7
[10]	Zheng J J, Fang X, Wen K, Li J Y, Ma Y, Liu M, An S, Li J L, Zalevsky Z, Gao P 2022 Opt. Express 30 27951 Google Scholar
[11]	Huang X S, Fan J C, Li L J, Liu H S, Wu R L, Wu Y, Wei L S, Mao H, Lal A, Xi P, Tang L Q, Zhang Y F, Liu Y M, Tan S, Chen L Y 2018 Nat. Biotechnol. 36 451 Google Scholar
[12]	Zhao W S, Zhao S Q, Li L J, Huang X S, Xing S J, Zhang Y L, Qiu G H, Han Z Q, Shang Y X, Sun D E, Shan C Y, Wu R L, Gu L S, Zhang S W, Chen R W, Xiao J, Mo Y Q, Wang J Y, Ji W, Chen X, Ding B Q, Liu Y M, Mao H, Song B L, Tan J B, Liu J, Li H Y, Chen L Y 2022 Nat. Biotechnol. 40 606 Google Scholar
[13]	Wen G, Li S M, Wang L B, Chen X H, Sun Z L, Liang Y, Jin X, Xing Y F, Jiu Y M, Tang Y G, Li H 2021 Light-Sci. Appl. 10 70 Google Scholar
[14]	Tu S J, Liu Q L, Liu X, Liu W J, Zhang Z M, Luo T J, Kuang C F, Liu X, Hao X 2020 Opt. Lett. 45 1567 Google Scholar
[15]	Dan D, Wang Z J, Zhou X, Lei M, Zhao T Y, Qian J, Yu X H, Yan S H, Min J W, Bianco P, Yao B L 2021 IEEE Photonics J. 13 3900411 Google Scholar
[16]	Wang Z J, Zhao T Y, Hao H W, Cai Y N, Feng K, Yun X, Liang Y S, Wang S W, Sun Y J, Bianco P R, Oh K, Lei M 2022 Adv. Photonics 4 026003 Google Scholar
[17]	Wang Z J, Zhao T Y, Cai Y A, Zhang J X, Hao H W, Liang Y S, Wang S W, Sun Y J, Chen T S, Bianco P R, Oh K, Lei M 2023 The Innovation 4 100425 Google Scholar
[18]	Wen G, Li S M, Liang Y, Wang L B, Zhang J, Chen X H, Jin X, Chen C, Tang Y G, Li H 2023 PhotoniX 4 19 Google Scholar
[19]	Müller M, Mönkemöller V, Hennig S, Hübner W, Huser T 2016 Nat. Commun. 7 10980 Google Scholar
[20]	Zhou L L, Yu B, Huang L L, Cao H Q, Lin D Y, Jing Y Y, Wali F, Qu J L 2022 ACS Appl. Nano Mater. 5 18742 Google Scholar
[21]	Descloux A, Grussmayer K S, Radenovic A 2019 Nat. Methods 16 918 Google Scholar

[1]	Luo Ze-Wei, Wu Ge, Chen Zhi, Deng Chi-Nan, Wan Rong, Yang Tao, Zhuang Zheng-Fei, Chen Tong-Sheng. Dual-channel structured illumination super-resolution quantitative fluorescence resonance energy transfer imaging. Acta Physica Sinica, 2023, 72(20): 208701. doi: 10.7498/aps.72.20230853
[2]	Ling Jin-Zhong, Guo Jin-Kun, Wang Yu-Cheng, Liu Xin, Wang Xiao-Rui. Research on spatial frequency shift super-resolution imaging based on evanescent wave illumination. Acta Physica Sinica, 2023, 72(22): 224202. doi: 10.7498/aps.72.20230934
[3]	Gao Zhao-Lin, Liu Rui-Hua, Wen Kai, Ma Ying, Li Jian-Lang, Gao Peng. Phase/fluorescence dual-mode microscopy imaging based on structured light illumination. Acta Physica Sinica, 2022, 71(24): 244203. doi: 10.7498/aps.71.20221518
[4]	Ge Yang-Yang, He Zhuo-Fen, Huang Li-Lin, Lin Dan-Ying, Cao Hui-Qun, Qu Jun-Le, Yu Bin. Flat-field multiplexed multifocal structured illumination super-resolution microscopy. Acta Physica Sinica, 2022, 71(4): 048704. doi: 10.7498/aps.71.20211712
[5]	. Flat-field multiplexed multifocal structured illumination super-resolution microscopy. Acta Physica Sinica, 2021, (): . doi: 10.7498/aps.70.20211712
[6]	Liu Kang, He Tao, Liu Tao, Li Guo-Qing, Tian Bo, Wang Jia-Yi, Yang Shu-Ming. Effect of laser illumination conditions on focusing performance of super-oscillatory lens. Acta Physica Sinica, 2020, 69(18): 184215. doi: 10.7498/aps.69.20200577
[7]	Qian Jia, Dang Shi-Pei, Zhou Xing, Dan Dan, Wang Zhao-Jun, Zhao Tian-Yu, Liang Yan-Sheng, Yao Bao-Li, Lei Ming. Fast structured illumination three-dimensional color microscopic imaging method based on Hilbert-transform. Acta Physica Sinica, 2020, 69(12): 128701. doi: 10.7498/aps.69.20200352
[8]	Zhang Jia, Samanta Soham, Wang Jia-Lin, Wang Lu-Wei, Yang Zhi-Gang, Yan Wei, Qu Jun-Le. Study on a novel probe for stimulated emission depletion Super-resolution Imaging of Mitochondria. Acta Physica Sinica, 2020, 69(16): 168702. doi: 10.7498/aps.69.20200171
[9]	Xiao Xiao, Du Shu-Man, Zhao Fu, Wang Jing, Liu Jun, Li Ru-Xin. Single-shot optical speckle imaging based on pseudothermal illumination. Acta Physica Sinica, 2019, 68(3): 034201. doi: 10.7498/aps.68.20181723
[10]	Yan Bo, Chen Li, Chen Shuang, Li Meng, Yin Yi-Min, Zhou Jiang-Ning. Structured illumination for two-dimensional laser induced fluorescence imaging to eliminate stray light interference. Acta Physica Sinica, 2019, 68(21): 218701. doi: 10.7498/aps.68.20190977
[11]	Tian Yuan, Ge Hao, Lu Ming-Hui, Chen Yan-Feng. Research advances in acoustic metamaterials. Acta Physica Sinica, 2019, 68(19): 194301. doi: 10.7498/aps.68.20190850
[12]	Liu Xiong-Bo, Lin Dan-Ying, Wu Qian-Qian, Yan Wei, Luo Teng, Yang Zhi-Gang, Qu Jun-Le. Recent progress of fluorescence lifetime imaging microscopy technology and its application. Acta Physica Sinica, 2018, 67(17): 178701. doi: 10.7498/aps.67.20180320
[13]	Zhao Tian-Yu, Zhou Xing, Dan Dan, Qian Jia, Wang Zhao-Jun, Lei Ming, Yao Bao-Li. Polarization control methods in structured illumination microscopy. Acta Physica Sinica, 2017, 66(14): 148704. doi: 10.7498/aps.66.148704
[14]	Lin Dan-Ying, Qu Jun-Le. Recent progress on super-resolution imaging and correlative super-resolution microscopy. Acta Physica Sinica, 2017, 66(14): 148703. doi: 10.7498/aps.66.148703
[15]	Zhang Chong-Lei, Xin Zi-Qiang, Min Chang-Jun, Yuan Xiao-Cong. Research progress of plasmonic structure illumination microscopy. Acta Physica Sinica, 2017, 66(14): 148701. doi: 10.7498/aps.66.148701
[16]	Liu Hong-Ji, Liu Shuang-Long, Niu Han-Ben, Chen Dan-Ni, Liu Wei. A super-resolution infrared microscopy based on a doughnut pump beam. Acta Physica Sinica, 2016, 65(23): 233601. doi: 10.7498/aps.65.233601
[17]	Zhao Ying-Chun, Zhang Xiu-Ying, Yuan Cao-Jin, Nie Shou-Ping, Zhu Zhu-Qing, Wang Lin, Li Yang, Gong Li-Ping, Feng Shao-Tong. Dark-field digital holographic microscopy by using vortex beam illumination. Acta Physica Sinica, 2014, 63(22): 224202. doi: 10.7498/aps.63.224202
[18]	Li Heng, Yu Bin, Chen Dan-Ni, Niu Han-Ben. Design and experimental demonstration of high-efficiency double-helix point spread function phase plate. Acta Physica Sinica, 2013, 62(12): 124201. doi: 10.7498/aps.62.124201
[19]	Zhi Shao-Tao, Zhang Hai-Jun, Zhang Dong-Xian. Super-resolution optical microscopic imaging method based on annular illumination with high numerical aperture. Acta Physica Sinica, 2012, 61(2): 024207. doi: 10.7498/aps.61.024207
[20]	Liang Gao-Feng, Zhao Qing, Chen Xin, Wang Chang-Tao, Zhao Ze-Yu, Luo Xian-Gang. Research of subwavelength grating based on multilayer films structure. Acta Physica Sinica, 2012, 61(10): 104203. doi: 10.7498/aps.61.104203

Catalog

Vol. 73, Issue 9 - 2024-05-05

Metrics

Abstract views: 330
PDF Downloads: 11
Cited By: 0

姓名
邮箱
手机号码
标题
留言内容
验证码

Search

Article

留言板