搜索

x

留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

基于希尔伯特变换的结构光照明快速三维彩色显微成像方法

千佳 党诗沛 周兴 但旦 汪召军 赵天宇 梁言生 姚保利 雷铭

引用本文:
Citation:

基于希尔伯特变换的结构光照明快速三维彩色显微成像方法

千佳, 党诗沛, 周兴, 但旦, 汪召军, 赵天宇, 梁言生, 姚保利, 雷铭

Fast structured illumination three-dimensional color microscopic imaging method based on Hilbert-transform

Qian Jia, Dang Shi-Pei, Zhou Xing, Dan Dan, Wang Zhao-Jun, Zhao Tian-Yu, Liang Yan-Sheng, Yao Bao-Li, Lei Ming
PDF
HTML
导出引用
  • 结构光照明显微是一种宽场显微技术, 可以实现超分辨成像和三维光切片成像. 基于HSV (色相、饱和度、明度)彩色空间的结构光照明全彩色三维光切片成像技术可以复原样品表面的真彩色信息, 但每一层光切片都需要采集3幅固定相移差的原始图像, 这对于需要多视场拼接的大尺寸样品而言, 图像采集数据量大、图像重构时间长的缺点就凸显出来. 鉴于此, 本文提出一种基于希尔伯特变换的结构光照明快速三维彩色显微成像方法, 只需在样品的每一层采集2幅原始图像便可以重构出该层的全彩色光切片图像, 因此图像采集量减少了1/3, 图像重构时间节约了约28%, 有效提高了彩色三维成像的效率和速度.
    As a wide-field microscopy, structured illumination microscopy (SIM) enables super-resolution and three-dimensional (3D) imaging. It has recently received lots of attention due to the advantages of high spatial resolution, short image recording time, and less photobleaching and phototoxicity. The SIM has found numerous important applications in time-lapse imaging of living tissues and cellular structures in the field of biomedical science. Color information is an important physical quantity describing the characteristics of living creatures and reflects the differences in its microstructure and optical property to some extent. Although HSV (hue, saturation, value) color space based structured illumination full-color 3D optical sectioning technique can recover the full color information on the surface of the samples without color distortion. However, for each optical sectioning, three raw images with fixed phase shift are required to calculate the sectioning images by the rootmean square (RMS) algorithm. This will dramatically increase the data acquisition time and data storage space, especially for a large-scaled sample that needs image stitching strategy. The image processing progress operated in HSV color space need to run the RMS algorithm three times in each channel of HSV space for every section, and transform the images between RGB (red-green-blue) space and HSV space twice. This will absolutely extend the data processing time and put forward higher requirements for computer hardware and software for data storage and processing. To this end, in this paper, a fast 3D color optical sectioning SIM algorithm based on Hilbert-transform is proposed. The Hilbert-transform has proved to be a powerful tool in digital signal and image processing and has successfully applied to the SIM. Here, only two raw images with structured illumination are needed to reconstruct a full-color optical sectioned image for each slice. This fast 3D color sectioning method has the advantage of insensitivity to phase-shift error and has better adaptability to noise, high quality color sectioning images can be obtained under the phase-shift error or noise disturbed environment. The image acquisition data are reduced by 1/3 and the color optical sectioning reconstruction time is saved by about 28%, this new method effectively improves the efficiency and speed for 3D color imaging and will bring a wider application range for SIM.
      通信作者: 姚保利, yaobl@opt.ac.cn ; 雷铭, ming.lei@mail.xjtu.edu.cn
    • 基金项目: 国家级-国家自然科学基金(91750106)
      Corresponding author: Yao Bao-Li, yaobl@opt.ac.cn ; Lei Ming, ming.lei@mail.xjtu.edu.cn
    [1]

    Liu F, Dong B, Liu X, Zheng Y, Zi J 2009 Opt. Express 17 16183Google Scholar

    [2]

    Shevtsova E, Hansson C, Janzen D H, Kjærandsen J 2011 Proc. Natl. Acad. Sci. USA 108 668Google Scholar

    [3]

    Kinoshita S, Yoshioka S 2005 Chem. Phys. Chem. 6 1442Google Scholar

    [4]

    Conchello J A, Lichtman J W 2005 Nat. Methods 2 920Google Scholar

    [5]

    Helmchen F, Denk W 2005 Nat. Methods 2 932Google Scholar

    [6]

    Michels J 2007 J. Microscopy 227 1Google Scholar

    [7]

    Mahou P, Vermot J, Beaurepaire E, Supatto W 2014 Nat. Methods 11 600Google Scholar

    [8]

    Dunn K W, Sandoval R M, Kelly K J, et al. 2002 Am. J. Physiol.-Cell Physiol. 28 C905Google Scholar

    [9]

    Gustafsson M G 2000 J. Microscopy 198 82Google Scholar

    [10]

    Shao L, Kner P, Rego E H, Gustafsson M G 2011 Nat. Methods 8 1044Google Scholar

    [11]

    Kner P, Chhun B B, Griffis E R, Winoto L, Gustafsson M G 2009 Nat. Methods 6 339Google Scholar

    [12]

    Neil M A, Juškaitis R, Wilson T 1997 Opt. Lett. 22 1905Google Scholar

    [13]

    Thomas B, Momany M, Kner P 2013 J. Opt. 15 094004Google Scholar

    [14]

    Mertz J 2011 Nat. Methods 8 811Google Scholar

    [15]

    Dan D, Lei M, Yao B, et al. 2013 Sci. Rep. 3 1116Google Scholar

    [16]

    Karadaglić D, Wilson T 2008 Micron 39 808Google Scholar

    [17]

    Patorski K, Trusiak M, Tkaczyk T 2014 Opt. Express 22 9517Google Scholar

    [18]

    Dan D, Yao B, Lei M 2014 Chin. Sci. Bull. 59 1291Google Scholar

    [19]

    Zhou X, Lei M, Dan D, Yao B, Qian J, Yan S, Yang Y, Min J, Peng T, Ye T 2015 PloS One 10Google Scholar

    [20]

    Qian J, Lei M, Dan D, Yao B, Zhou X, Yang Y, Yan S, Min J, Yu X 2015 Sci. Rep. 5 14513Google Scholar

    [21]

    Qian J, Dang S, Wang Z, Zhou X, Dan D, Yao B, Tong Y, Yang H, Lu Y, Chen Y, Yang X, Bai M, Lei M 2019 Opt. Express 27 4845Google Scholar

    [22]

    周兴 2018 博士学位论文 (西安: 西安交通大学)

    Zhou X 2018 Ph. D. Dissertation (Xi’an: Xi’an Jiaotong University) (in Chinese)

    [23]

    Nayar S K, Nakagawa Y 1990 Proceedings, IEEE International Conference on Robotics and Automation Cincinnati, USA, May 13–18, 1990 p218

    [24]

    Nayar S K, Nakagawa Y 1994 IEEE Trans. Pattern Anal. Mach. Intell. 16 824Google Scholar

  • 图 1  HT-COS算法流程图和HSV-RMS算法流程图对比 (a) HT-COS算法流程图; (b) HSV-RMS算法流程图

    Fig. 1.  Flowchart diagram comparison between HT-COS algorithm and HSV-RMS algorithm: (a) HT-COS algorithm; (b) HSV-RMS algorithm.

    图 2  相移误差对三维光切片图像的影响 (a)含有离焦背景的宽场图像; (b)结构光照明图像; (c)−(f)在不同的相移误差下, HSV-RMS算法所复原的彩色光切片图像; (g)−(j) 不同相移误差下, HT-COS算法所复原的彩色光切片图像

    Fig. 2.  Effect of phase-shift error on optical sectioning images: (a) Wide-field image with defocused background; (b) structured illumination image; (c)−(f) three-dimensional (3D) color optical sectioning images processed by HSV-RMS algorithm under different phase-shift errors; (g)−(j) 3D color optical sectioning images processed by HT-COS algorithm under different phase-shift errors.

    图 3  高斯噪声对光切片重构图像的影响 (a)−(c) 高斯噪声方差为0.01时的结构光照明图像、HSV-RMS算法得到的光切片图像及HT-COS算法得到的光切片图像; (d)−(f) 高斯噪声方差为0.03时的结果; (g)−(i) 高斯噪声方差为0.05时的结果

    Fig. 3.  Influence of Gaussian noise on the reconstructed optical sectioning images. Structured illumination image, optical sectioning images calculated by the HSV-RMS algorithm and HT-COS algorithm, respectively, under the conditions of the Gaussian noise with variances of (a)−(c) 0.01, (d)−(f) 0.03, and (g)−(i) 0.05.

    图 4  HSV-RMS算法和HT-COS算法的色彩复原保真度比较 (a)−(c)分别为原始图像、HSV-RMS算法处理后的光切片图像及HT-COS算法处理后的光切片图像; (d)−(f) 3幅图像各色块内的RGB值

    Fig. 4.  Comparison of color restoration fidelity between HSV-RMS algorithm and HT-COS algorithm: (a)−(c) Raw image, optical sectioning image calculated by HSV-RMS algorithm and optical sectioning image calculated by HT-COS algorithm, respectively; (d)−(f) RGB values for the four different regions of each image.

    图 5  结构光照明彩色光切片实验光路及系统图 (a) 系统光路原理图; (b)系统实物图

    Fig. 5.  Schematic diagram of structured illumination color optical sectioning system: (a) Light-path diagram; (b) apparatus diagram

    图 6  HSV-RMS算法和HT-COS算法重构的花粉彩色三维光切片图像效果对比 (a) HSV-RMS算法重构的三维图像; (b) HT-COS算法重构的三维图像; (c) 图(a)紫色方框区域的放大图像; (d) 图(b)绿色方框区域内的放大图像; (e) 图(c)中蓝色虚线和图(d)中红色实线上的强度分布; 标尺: 30 μm

    Fig. 6.  Comparison of reconstructed result of pollen grain between HSV-RMS algorithm and HT-COS algorithm: (a) 3D reconstructed color image from HSV-RMS algorithm; (b) 3D reconstructed color image from HT-COS algorithm; (c) the enlarged image in the purple rectangular box in panel (a); (d) the enlarged image in the green rectangular box in (b); (e) normalized intensity distribution of the line-scan in panel (c) and (d), i.e. the “root-shaped” structure. Scale bar: 30 μm

    图 7  一种中华虎甲背部的三维彩色SIM成像结果 (a) 该中华虎甲样品完整三维图像的二维最大值投影, 使用4 ×, NA = 0.2物镜拍摄, 共拼接84个视场, 单视场轴向扫描350层; (b) 样品轴向进行三维叠加重构的示意图, 每一层都是经过HSV-RMS算法处理后的光切片图; (c) 图(a)中红色箭头所指区域局部放大的三维光切片最大值投影图像, 使用HSV-RMS算法进行图像处理, 20 ×, NA = 0.45物镜拍摄; (d) 图(c)的三维形貌分布; (e) 样品轴向进行三维叠加重构的示意图, 每一层都是经过HT-COS算法处理后的光切片图; (f)图(a)中红色箭头所指区域局部放大的三维光切片最大值投影图像, 使用HT-COS算法进行图像处理, 20 ×, NA = 0.45物镜拍摄; (g) 图(f)的三维形貌分布

    Fig. 7.  3D color imaging result of a Chinese tiger beetle: (a) Maximum intensity projection image of the tiger beetle under 4 ×, NA = 0.2 objective lens, the 3D volume is rendered from 84 data sets stitching and sliced 350 layers; (b) schematic diagram of 3D reconstruction in axial direction after imaging processing with HSV-RMS algorithm; (c) maximum intensity projection images of the area pointed by the red arrow in panel (a) processed with HSV-RMS algorithm. The images are captured under 20 ×, NA = 0.45 objective lens; (d) 3D height map of panel (c); (e) schematic diagram of 3D reconstruction in axial direction after imaging processing with HT-COS algorithm; (f) maximum intensity projection images of the area pointed by the red arrow in (a) processed with HT-COS algorithm. The images are captured under 20 ×, NA = 0.45 objective lens; (g) the 3D height map of panel (f).

    表 1  两种SIM彩色光切片算法的性能比较

    Table 1.  Performance comparison of two algorithms for color optical sectioning SIM.

    视场个数图像采集总时间/s图像处理总时间/s图像数据容量/GB三维数据总像素数/pixels
    HSV-RMS算法8417671360810001010
    HT-COS算法84147297446701010
    下载: 导出CSV
  • [1]

    Liu F, Dong B, Liu X, Zheng Y, Zi J 2009 Opt. Express 17 16183Google Scholar

    [2]

    Shevtsova E, Hansson C, Janzen D H, Kjærandsen J 2011 Proc. Natl. Acad. Sci. USA 108 668Google Scholar

    [3]

    Kinoshita S, Yoshioka S 2005 Chem. Phys. Chem. 6 1442Google Scholar

    [4]

    Conchello J A, Lichtman J W 2005 Nat. Methods 2 920Google Scholar

    [5]

    Helmchen F, Denk W 2005 Nat. Methods 2 932Google Scholar

    [6]

    Michels J 2007 J. Microscopy 227 1Google Scholar

    [7]

    Mahou P, Vermot J, Beaurepaire E, Supatto W 2014 Nat. Methods 11 600Google Scholar

    [8]

    Dunn K W, Sandoval R M, Kelly K J, et al. 2002 Am. J. Physiol.-Cell Physiol. 28 C905Google Scholar

    [9]

    Gustafsson M G 2000 J. Microscopy 198 82Google Scholar

    [10]

    Shao L, Kner P, Rego E H, Gustafsson M G 2011 Nat. Methods 8 1044Google Scholar

    [11]

    Kner P, Chhun B B, Griffis E R, Winoto L, Gustafsson M G 2009 Nat. Methods 6 339Google Scholar

    [12]

    Neil M A, Juškaitis R, Wilson T 1997 Opt. Lett. 22 1905Google Scholar

    [13]

    Thomas B, Momany M, Kner P 2013 J. Opt. 15 094004Google Scholar

    [14]

    Mertz J 2011 Nat. Methods 8 811Google Scholar

    [15]

    Dan D, Lei M, Yao B, et al. 2013 Sci. Rep. 3 1116Google Scholar

    [16]

    Karadaglić D, Wilson T 2008 Micron 39 808Google Scholar

    [17]

    Patorski K, Trusiak M, Tkaczyk T 2014 Opt. Express 22 9517Google Scholar

    [18]

    Dan D, Yao B, Lei M 2014 Chin. Sci. Bull. 59 1291Google Scholar

    [19]

    Zhou X, Lei M, Dan D, Yao B, Qian J, Yan S, Yang Y, Min J, Peng T, Ye T 2015 PloS One 10Google Scholar

    [20]

    Qian J, Lei M, Dan D, Yao B, Zhou X, Yang Y, Yan S, Min J, Yu X 2015 Sci. Rep. 5 14513Google Scholar

    [21]

    Qian J, Dang S, Wang Z, Zhou X, Dan D, Yao B, Tong Y, Yang H, Lu Y, Chen Y, Yang X, Bai M, Lei M 2019 Opt. Express 27 4845Google Scholar

    [22]

    周兴 2018 博士学位论文 (西安: 西安交通大学)

    Zhou X 2018 Ph. D. Dissertation (Xi’an: Xi’an Jiaotong University) (in Chinese)

    [23]

    Nayar S K, Nakagawa Y 1990 Proceedings, IEEE International Conference on Robotics and Automation Cincinnati, USA, May 13–18, 1990 p218

    [24]

    Nayar S K, Nakagawa Y 1994 IEEE Trans. Pattern Anal. Mach. Intell. 16 824Google Scholar

  • [1] 杨浩智, 聂梦娇, 马光鹏, 曹慧群, 林丹樱, 屈军乐, 于斌. 基于DMD的快速超分辨晶格结构光照明显微研究. 物理学报, 2024, 0(0): . doi: 10.7498/aps.73.20240216
    [2] 付亚鹏, 孙乾东, 李博艺, 他得安, 许凯亮. 基于RCA阵列三维超快超声血流成像方法仿真研究. 物理学报, 2023, 72(7): 074302. doi: 10.7498/aps.72.20222106
    [3] 罗泽伟, 武戈, 陈挚, 邓驰楠, 万蓉, 杨涛, 庄正飞, 陈同生. 双通道结构光照明超分辨定量荧光共振能量转移成像系统. 物理学报, 2023, 72(20): 208701. doi: 10.7498/aps.72.20230853
    [4] 葛阳阳, 何灼奋, 黄黎琳, 林丹樱, 曹慧群, 屈军乐, 于斌. 平场复用多焦点结构光照明超分辨显微成像. 物理学报, 2022, 71(4): 048704. doi: 10.7498/aps.71.20211712
    [5] 高兆琳, 刘瑞桦, 温凯, 马英, 李建郎, 郜鹏. 结构光照明相位/荧光双模式显微技术. 物理学报, 2022, 71(24): 244203. doi: 10.7498/aps.71.20221518
    [6] 胡金虎, 林丹樱, 张炜, 张晨爽, 屈军乐, 于斌. 结合虚拟单像素成像解卷积的双边照明光片荧光显微技术. 物理学报, 2022, 71(2): 028701. doi: 10.7498/aps.71.20211358
    [7] 钟虓䶮, 李卓. 原子尺度材料三维结构、磁性及动态演变的透射电子显微学表征. 物理学报, 2021, 70(6): 066801. doi: 10.7498/aps.70.20202072
    [8] 葛阳阳, 于斌. 平场复用多焦点结构光照明超分辨显微成像研究. 物理学报, 2021, (): . doi: 10.7498/aps.70.20211712
    [9] 冯帅, 常军, 胡瑶瑶, 吴昊, 刘鑫. 偏振成像激光雷达与短波红外复合光学接收系统设计与分析. 物理学报, 2020, 69(24): 244202. doi: 10.7498/aps.69.20200920
    [10] 王佳林, 严伟, 张佳, 王璐玮, 杨志刚, 屈军乐. 受激辐射损耗超分辨显微成像系统研究的新进展. 物理学报, 2020, 69(10): 108702. doi: 10.7498/aps.69.20200168
    [11] 闫博, 陈力, 陈爽, 李猛, 殷一民, 周江宁. 结构光照明技术在二维激光诱导荧光成像去杂散光中的应用. 物理学报, 2019, 68(21): 218701. doi: 10.7498/aps.68.20190977
    [12] 范爽, 张亚萍, 王帆, 高云龙, 钱晓凡, 张永安, 许蔚, 曹良才. 面向真彩色三维显示的分层角谱算法和Gerchberg-Saxton算法研究. 物理学报, 2018, 67(9): 094203. doi: 10.7498/aps.67.20172464
    [13] 赵天宇, 周兴, 但旦, 千佳, 汪召军, 雷铭, 姚保利. 结构光照明显微中的偏振控制. 物理学报, 2017, 66(14): 148704. doi: 10.7498/aps.66.148704
    [14] 张崇磊, 辛自强, 闵长俊, 袁小聪. 表面等离激元结构光照明显微成像技术研究进展. 物理学报, 2017, 66(14): 148701. doi: 10.7498/aps.66.148701
    [15] 李丹, 张宝龙, 郭海成. 垂直向列型彩色滤光膜硅覆液晶微显示器的三维光学建模. 物理学报, 2015, 64(14): 140701. doi: 10.7498/aps.64.140701
    [16] 赵应春, 张秀英, 袁操今, 聂守平, 朱竹青, 王林, 李杨, 贡丽萍, 冯少彤. 基于涡旋光照明的暗场数字全息显微方法研究. 物理学报, 2014, 63(22): 224202. doi: 10.7498/aps.63.224202
    [17] 支绍韬, 章海军, 张冬仙. 基于大数值孔径环形光锥照明的超分辨光学显微成像方法研究. 物理学报, 2012, 61(2): 024207. doi: 10.7498/aps.61.024207
    [18] 张宝龙, 李丹, 戴凤智, 杨世凤, 郭海成. 彩色滤光膜硅覆液晶微显示器的三维光学建模. 物理学报, 2012, 61(4): 040701. doi: 10.7498/aps.61.040701
    [19] 林浩铭, 邵永红, 屈军乐, 尹 君, 陈思平, 牛憨笨. 散斑照明宽场荧光层析显微成像技术研究. 物理学报, 2008, 57(12): 7641-7649. doi: 10.7498/aps.57.7641
    [20] 陈冠英, 历树忠. 一种新的体视假彩色显微镜成像机理分析. 物理学报, 1999, 48(1): 23-30. doi: 10.7498/aps.48.23
计量
  • 文章访问数:  7908
  • PDF下载量:  171
  • 被引次数: 0
出版历程
  • 收稿日期:  2020-03-09
  • 修回日期:  2020-04-02
  • 刊出日期:  2020-06-20

/

返回文章
返回