-
Single-molecule localization technology has been widely used in single-particle tracking and super-resolution imaging of biological samples, as it can bypass the diffraction limit of optical systems. Multi-channel single-molecule localization uses multiple imaging channels to simultaneously track different targets or perform multi-color super-resolution imaging, and can also improve the axial depth of single-particle tracking or achieve higher localization precision and density for super-resolution imaging. However, the difference between images in each channel can affect collaborative localization or quantitative analysis, so image registration is a key step in its image data preprocessing. Moreover, due to the high precision of single-molecule localization, its requirements for multi-channel image registration accuracy are also high. Existing technologies generally use control point-based registration methods and often use complicated and precise methods to obtain fiducial images for locating control point pairs to achieve high-precision image registration, which involves high sample or experimental equipment requirements and is difficult to directly extend to other systems. Therefore, developed in this work, is a high-precision image registration method that can directly use randomly distributed fluorescent beads as fiducial samples based on local nonlinear transformation and elimination of mismatched points. By monitoring and iteratively filtering control points in the process of feature matching and transformation model parameter estimation to eliminate control point pairs that are not accurately matched due to inaccurate or poor precision of single-molecule localization, the adverse effects on accurate acquisition and precise matching of control points when using randomly distributed fluorescent beads as fiducial samples are eliminated. At the same time, a second-order polynomial fitting based on local weighted mean is used for estimating the transformation model parameter to better adapt to the existence of local nonlinear deformation between different channels. The results show that using this method only requires three iterations to find and eliminate control point pairs that are not accurately located and matched, thereby achieving more accurate transformation model parameter and improving the registration accuracy by an order of magnitude, achieving a registration accuracy of about 6 nm in a complex dual-channel single-molecule localization imaging system based on orthogonal astigmatism.
-
Keywords:
- single molecule localization /
- multi-channel imaging /
- image registration /
- elimination of mismatched points
[1] Diezmann A V, Shechtman Y, Moerner W E 2017 Chem. Rev. 117 7244
Google Scholar
[2] Saxton M J, Jacobson K 1997 Annu. Rev. Biophys. Biomol. Struct. 26 373
Google Scholar
[3] Deich J, Judd E M, Mcadams H H, Moerner W E 2004 Proc. Natl. Acad. Sci. U. S. A. 101 15921
Google Scholar
[4] Rust M J, Bates M, Zhuang X 2006 Nat. Methods 3 793
Google Scholar
[5] Sigal Y M, Zhou R, Zhuang X 2018 Science 361 880
Google Scholar
[6] Driouchi A, Gray-Owen S D, Yip C M 2022 J. Biol. Chem. 298 102448
Google Scholar
[7] Louis B, Camacho R, Bresolí-Obach R, Abakumov S, Vandaele J, Kudo T, Masuhara H, Scheblykin I, Hofkens J, Rocha S 2020 Opt. Express 28 28656
Google Scholar
[8] Albrecht D, Winterflood C M, Ewers H 2015 Methods Appl. Fluoresc. 3 024001
Google Scholar
[9] Bates M, Dempsey G T, Chen K H, Zhuang X 2012 ChemPhysChem 13 99
Google Scholar
[10] Lehmann M, Gottschalk B, Puchkov D, Schmieder P, Schwagerus S, Hackenberger P, Haucke V, Schmoranzer J 2015 Angew. Chem. Int. Ed. 54 13230
Google Scholar
[11] Gu L, Sheng Y, Chen Y, Chang H, Zhang Y, Lv P, Ji W, Xu T 2014 Biophys. J. 106 2443
Google Scholar
[12] Min J, Holden S J, Carlini L, Unser M, Manley S, Ye J C 2014 Biomed. Opt. Express 5 3935
Google Scholar
[13] 林丹樱, 武泽凯, 于斌, 黄黎琳, 张潇, 屈军乐 2022 物理学报 71 128701
Google Scholar
Lin D Y, Wu Z K, Yu B, Huang L L, Zhang X, Qu J L 2022 Acta Phys. Sin. 71 128701
Google Scholar
[14] Deschout H, Shivanandan A, Annibale P, Scarselli M, Radenovic A 2014 Histochem. Cell Biol. 142 5
Google Scholar
[15] Paul S, Pati U C 2021 Int. J. Remote Sens. 42 5396
Google Scholar
[16] Churchman L S, Okten Z, Rock R S, Dawson J F, Spudich J A 2005 Proc. Natl. Acad. Sci. U. S. A. 102 1419
Google Scholar
[17] Gahlmann A, Ptacin J L, Grover G, Quirin S, von Diezmann A R S, Lee M K, Backlund M P, Shapiro L, Piestun R, Moerner W E 2013 Nano Lett. 13 987
Google Scholar
[18] Huang B, Wang W Q, Bates M, Zhuang X 2008 Science 319 810
Google Scholar
[19] Goshtasby A 1988 Image Vison Comput. 6 255
Google Scholar
[20] Zagorchev L, Goshtasby A 2006 IEEE Trans. Image Process. 15 529
Google Scholar
[21] Huang F, Sirinakis G, Allgeyer E S, Toomre D, Booth M J, Bewersdorf J 2016 Cell 166 1028
Google Scholar
[22] Fischler M A, Bolles R C 1981 Commun. ACM 24 381
Google Scholar
[23] 张岩, 孙世宇, 胡永江, 李建增, 范聪 2018 电子与信息学报 40 928
Google Scholar
Zhang Y, Sun S Y, Hu Y J, Li J Z, Fan C 2018 J. Electron. Inf. Technol. 40 928
Google Scholar
[24] 赖焕杰, 孟祥印, 肖世德, 胡锴沣, 李召鑫 2023 传感器与微系统 42 135
Lai H J, Meng X Y, Xiao S D, Hu K F, Li Z X 2023 Transducer Microsys. Technol. 42 135
-
图 1 正交像散单分子定位及其图像配准需求示意图 (a)正交像散系统光路; (b)不同轴向位置物点对应的像散图像对; (c)正交像散导致的双通道图像形变及预期配准效果. ND, 衰减片; L, 透镜; M, 反射镜; DM, 二向色镜; OL, 物镜; F, 滤光片; A, 光阑; BS, 分束器; CL, 柱透镜; EMCCD, 电子倍增电荷耦合器件
Figure 1. Illustration of the orthogonal astigmatism-based single molecule localization method and its requirements on image registration: (a) Optical path of the orthogonal astigmatism system; (b) astigmatic image pairs corresponding to object points at different axial positions; (c) dual-channel image distortion caused by orthogonal astigmatism and expected registration result. ND, attenuator; L, lens; M, mirror; DM, dichroic mirror; OL, objective lens; F, filter; A, aperture; BS, beam splitter; CL, cylindrical lens; EMCCD, electron-multiplying charge-coupled device.
图 2 基于基准配准误差监测的误匹配点剔除(EMP-FRE)算法的基本原理和算法流程示意图 (a)基本原理; (b)算法流程图. I, 图像; P, 定位点; T, 配准参数矩阵; FRE, 基准配准误差; E, 误差阈值; 下标l/r, 左/右通道; 下标i, 控制点编号
Figure 2. Schematic diagram (a) and algorithm flowchart (b) of the EMP-FRE (i.e., elimination of mismatched points based on fiducial registration error monitoring) algorithm. I, image; P, localizations; T, registration parameter matrix; FRE, fiducial registration error; E, error threshold; subscript l/r, left/right channel; subscript i, control point number.
图 3 变换模型选择和误匹配点剔除对配准效果的影响 (a)左通道参考图像; (b)右通道待配准图像; (c) LWM法配准后的右通道图像; (d)采用不同模型配准后的定位点; (e)采用不同模型配准后的平均FRE; (f), (g)采用LWM模型和EMP-FRE算法的X, Y方向配准误差统计. 统计数据来自5帧不同视野荧光珠图像的140对定位点(控制点)
Figure 3. Effects of transformation model selection and mismatched point elimination on registration performance: (a) Left channel, reference image; (b) right channel, image to be registered; (c) right channel, image after LWM registration; (d) localizations registered using different models; (e) average FRE after registration using different models; (f), (g) statistical data on registration errors in the X and Y directions using the LWM model and EMP-FRE algorithm, obtained from 140 pairs of localizations (control points) from 5 different field of view fluorescent bead images.
图 4 实验荧光珠图像的配准结果对比 (a)传统LWM的配准结果; (b)结合LWM和EMP-FRE的配准结果; (c), (d)蓝色方框的放大图; (e), (f)红色方框的放大图 伪彩色: 紫色代表左通道图像, 绿色代表配准后的右通道图像, 两者准确重合时为灰白色
Figure 4. Comparison of the registration results of experimental fluorescent bead images: (a) Registered image using traditional LWM; (b) registered image using LWM+EMP-FRE; (c), (d) enlarged views of the blue box; (e), (f) enlarged views of red box. Pseudo color: purple represents the left channel image, green represents the registered right channel image, and when they are accurately overlapped, the color should be grayish white.
图 5 平行线模拟样品的超分辨重构结果比较 (a)不同情形数据的重构图像: 1-无误差数据; 2-有误差未配准数据; 3-AT配准数据; 4-2PF配准数据; 5-PWL配准数据; 6-LWM配准数据; 7-LWM+EMP-FRE配准数据; (b)—(d)图(a)中3个白色方框处对应的截面轮廓曲线
Figure 5. Comparison of super-resolution reconstruction results for the simulated parallel line sample: (a) Reconstructed images from data in different cases: 1-error free data; 2-unregistered data with errors; 3-AT registered data; 4-2PF registered data; 5-PWL registered data; 6-LWM registered data; 7-LWM+EMP-FRE registered data; (b)–(d) the corresponding cross-section profiles of the three white boxes in panel (a).
图 6 微管实验数据的超分辨重构结果比较 (a)宽场荧光图像; (b), (c)图(a)中白色方框区域的数据分别采用LWM法和LWM+EMP-FRE法配准后的重构结果; (d)图(b)和(c)中红色和蓝色方框处对应的截面轮廓曲线
Figure 6. Comparison of super-resolution reconstruction results for microtubule experimental data: (a) Wide-field fluorescence image; (b), (c) reconstructed images from data of the white box region in panel (a), registered using LWM and LWM+EMP-FRE, respectively; (d) corresponding cross-section profiles of the red and blue boxes in panel (b) and (c), respectively.
表 1 图3(a)和图3(b)荧光珠图像的EMP-FRE迭代过程及利用剩余控制点对进行LWM配准所得的平均基准配准误差
Table 1. The EMP-FRE iteration process of the fluorescent bead images in Fig. 3(a), (b), and the average FRE obtained through LWM registration using residual control points.
迭代次数 剔除控制点对标号 平均基准配准误差/ nm 0 — 29.3 1 3, 4, 6, 7 11.7 2 1, 2, 3, 4, 5 3.4 3 2, 3, 4, 5 2.9 
DownLoad: CSV
表 2 误差阈值对迭代和平均配准误差的影响
Table 2. Effect of error threshold on iterative and average registration errors.
误差阈值/nm 每次迭代剔除的控制点对数量 每次迭代后的平均配准误差/nm 1 2 3 4 1 2 3 4 10 4 16 10 9 11.7 5.8 5.6 5.4 20 4 9 4 — 11.7 3.5 2.9 — 30 4 5 4 — 11.7 3.4 2.9 — 40 4 3 — — 11.7 10.4 — — 50 4 2 — — 11.7 11.6 — —
DownLoad: CSV
表 3 误匹配点剔除算法的效果及效率对比
Table 3. Comparison of the effectiveness and efficiency of algorithms for eliminating mismatched points.
误匹配点剔除算法 迭代成功率 迭代成功所需的
平均迭代次数RANSAC 0.84 165 RSCFDI 0.49 217 RASCFDI+
RANSAC2 match0.49 260
DownLoad: CSV
表 4 平行线模拟样品重构图像的线间距和对比度定量比较
Table 4. Quantitative comparison of line spacing and contrast in reconstructed images of the simulated parallel line sample.
线间距/nm 对比度/% b c d 平均 b c d 平均 Control 58.6 59.4 57.2 58.4 82.1 87.2 76.2 81.8 LWM 55.2 58.8 55.8 56.6 77.7 83.3 48.5 69.8 LWM+EMP-FRE 57.2 58.4 59.0 58.2 74.7 79.4 82.1 78.7
DownLoad: CSV
-
[1] Diezmann A V, Shechtman Y, Moerner W E 2017 Chem. Rev. 117 7244
Google Scholar
[2] Saxton M J, Jacobson K 1997 Annu. Rev. Biophys. Biomol. Struct. 26 373
Google Scholar
[3] Deich J, Judd E M, Mcadams H H, Moerner W E 2004 Proc. Natl. Acad. Sci. U. S. A. 101 15921
Google Scholar
[4] Rust M J, Bates M, Zhuang X 2006 Nat. Methods 3 793
Google Scholar
[5] Sigal Y M, Zhou R, Zhuang X 2018 Science 361 880
Google Scholar
[6] Driouchi A, Gray-Owen S D, Yip C M 2022 J. Biol. Chem. 298 102448
Google Scholar
[7] Louis B, Camacho R, Bresolí-Obach R, Abakumov S, Vandaele J, Kudo T, Masuhara H, Scheblykin I, Hofkens J, Rocha S 2020 Opt. Express 28 28656
Google Scholar
[8] Albrecht D, Winterflood C M, Ewers H 2015 Methods Appl. Fluoresc. 3 024001
Google Scholar
[9] Bates M, Dempsey G T, Chen K H, Zhuang X 2012 ChemPhysChem 13 99
Google Scholar
[10] Lehmann M, Gottschalk B, Puchkov D, Schmieder P, Schwagerus S, Hackenberger P, Haucke V, Schmoranzer J 2015 Angew. Chem. Int. Ed. 54 13230
Google Scholar
[11] Gu L, Sheng Y, Chen Y, Chang H, Zhang Y, Lv P, Ji W, Xu T 2014 Biophys. J. 106 2443
Google Scholar
[12] Min J, Holden S J, Carlini L, Unser M, Manley S, Ye J C 2014 Biomed. Opt. Express 5 3935
Google Scholar
[13] 林丹樱, 武泽凯, 于斌, 黄黎琳, 张潇, 屈军乐 2022 物理学报 71 128701
Google Scholar
Lin D Y, Wu Z K, Yu B, Huang L L, Zhang X, Qu J L 2022 Acta Phys. Sin. 71 128701
Google Scholar
[14] Deschout H, Shivanandan A, Annibale P, Scarselli M, Radenovic A 2014 Histochem. Cell Biol. 142 5
Google Scholar
[15] Paul S, Pati U C 2021 Int. J. Remote Sens. 42 5396
Google Scholar
[16] Churchman L S, Okten Z, Rock R S, Dawson J F, Spudich J A 2005 Proc. Natl. Acad. Sci. U. S. A. 102 1419
Google Scholar
[17] Gahlmann A, Ptacin J L, Grover G, Quirin S, von Diezmann A R S, Lee M K, Backlund M P, Shapiro L, Piestun R, Moerner W E 2013 Nano Lett. 13 987
Google Scholar
[18] Huang B, Wang W Q, Bates M, Zhuang X 2008 Science 319 810
Google Scholar
[19] Goshtasby A 1988 Image Vison Comput. 6 255
Google Scholar
[20] Zagorchev L, Goshtasby A 2006 IEEE Trans. Image Process. 15 529
Google Scholar
[21] Huang F, Sirinakis G, Allgeyer E S, Toomre D, Booth M J, Bewersdorf J 2016 Cell 166 1028
Google Scholar
[22] Fischler M A, Bolles R C 1981 Commun. ACM 24 381
Google Scholar
[23] 张岩, 孙世宇, 胡永江, 李建增, 范聪 2018 电子与信息学报 40 928
Google Scholar
Zhang Y, Sun S Y, Hu Y J, Li J Z, Fan C 2018 J. Electron. Inf. Technol. 40 928
Google Scholar
[24] 赖焕杰, 孟祥印, 肖世德, 胡锴沣, 李召鑫 2023 传感器与微系统 42 135
Lai H J, Meng X Y, Xiao S D, Hu K F, Li Z X 2023 Transducer Microsys. Technol. 42 135
DownLoad:
Catalog
Metrics
- Abstract views: 2845
- PDF Downloads: 146
- Cited By: 0
