-
Ultrafast ultrasound localization microscopy (uULM) has broken through the fundamental acoustic diffraction limit by accumulating thousands of sub-wavelength microbubble localisation points and improved the spatial resolution by more than one order of magnitude, which is conducive to clinical diagnosis. By localizing individually injected microbubbles and tracking their movement with a subwavelength resolution, the vasculature microscopy can be achieved with micrometer scale. However, the reconstruction of a uULM image often requires tens or even hundreds of seconds of continuous long-range image acquisition, which limits its clinical application. In order to solve this problem, a generative adversarial network (GAN) based deep learning method is proposed to reconstruct the super-resolution ultrasound localization microscopy. In vivo uULM ultrasound datasets are used to train the network to reconstruct dense vascular networks via localized microbubbles. This approach is validated by using another in-vivo dataset obtained in a rat brain. Results show that GAN based ultrafast ultrasound localization microscopy (GAN-uULM) can resolve micro vessels smaller than 10 μm. Besides, GAN-uULM is able to distinguish small vessels that cannot be continuously reconstructed by using a standard uULM reconstruction method. Saturation parameter based on counting the number of explored pixels is used to evaluate the reconstruction quality. The proposed reconstruction approach reduces the data requirement by half and thus significantly accelerates the uULM imaging. It is illustrasted that for a dataset of 292 s ultrafast acquisition, the saturation of standard uULM image is 33%, while that of GAN-uULM can reach 46%. Fourier ring correlation (FRC) method is utilized to measure the spatial resolution in uULM. Resolutions of the images obtained by standard uULM and GAN-ULM are 7.8 μm and 8.9 μm, respectively. In conclusion, the developed deep learning model is able to connect trajectories with less computational complexity and avoids manual tuning and trajectory screening, providing an effective solution for accelerating ultrasound localization microscopy. -
Keywords:
- super-resolution /
- ultrasonic localization microscopy /
- convolutional neural network /
- generative adversarial network
[1] 钟传钰, 郑元义 2021 中国医学影像技术 37 1799Google Scholar
Zhong C Y, Zheng Y Y 2021 Chin. J. Med. Imaging Technol. 37 1799Google Scholar
[2] 王宇森, 陶鸿根 1991 中华内分泌代谢杂志 7 2
Wang Y S, Tao H G 1991 Chin. J. Endocrinol. Metab. 7 2
[3] Chugh B P, Lerch J P, Yu L X, Pienkowski M, Harrison R V, Henkelman R M, Sled J G 2009 Neuroimage 47 1312Google Scholar
[4] Huang C H, Chen C C V, Siow T Y, Hsu S H S, Hsu Y H, Jaw F S, Chang C 2013 PLoS One 8 e78186Google Scholar
[5] Hong G, Lee J C, Robinson J T, Raaz U, Xie L M, Huang, N F, Cooke J P, Dai H J 2012 Nat. Med. 18 1841Google Scholar
[6] Yao J, Wang L, Yang J M, Maslov K I, Wong T T W, Li L, Huang C H, Zou J, Wang L V 2015 Nat. Methods 12 407Google Scholar
[7] O"Reilly M A, Hynynen K 2013 Med. Phys. 40 110701Google Scholar
[8] Jiang C, Li Y, Xu K, Ta D 2021 IEEE Trans. Ultrason. Ferroelectr. Freq. Control 68 72Google Scholar
[9] 臧佳琦, 许凯亮, 韩清见, 陆起涌, 梅永丰, 他得安 2021 物理学报 70 114304Google Scholar
Zang J Q, Xu K L, Han Q J, Lu Q Y, Mei Y F, Ta D A 2021 Acta Phys. Sin. 70 114304Google Scholar
[10] Sui Y H, Yan S Y, Zang J Q, Liu X, Ta D A, Wang W Q, Xu K L 2021 IEEE International Ultrasonics Symposium (IUS)
[11] Sui Y H, Yan S Y, Yu J J, Song J P, Ta D A, Wang W Q, Xu K L 2022 IEEE Trans. Ultrason. Ferroelectr. Freq. Control 69 2425Google Scholar
[12] Couture O, Bannouf S, Montaldo G, Aubry J F, Fink M 2009 Ultrasound Med. Biol. 35 1908Google Scholar
[13] Viessmann O M, Eckersley R J, Christensen-Jeffries K, Tang M X, Dunsby C 2013 Phys. Med. Biol. 58 6447Google Scholar
[14] Desailly Y, Couture O, Fink M, Tanter M 2013 Appl. Phys. Lett. 103 189Google Scholar
[15] Errico C, Pierre J, Pezet S, Desailly Y, Lenkei Z, Couture O, Tanter M 2015 Nature 527 499Google Scholar
[16] Fanglue L, Shelton S E, Espíndola D, Rojas J D, Gianmarco P, Dayton P A 2017 Theranostics 7 196Google Scholar
[17] Xu K L, Guo X Y, Sui Y H, Hingot V, Couture O, Ta D A, Wang W Q 2021 IEEE International Ultrasonics Symposium (IUS)
[18] 郁钧瑾, 郭星奕, 隋怡晖, 宋剑平, 他得安, 梅永丰, 许凯亮 2022 物理学报 71 174302Google Scholar
Yu J J, Guo X Y, Sui Y H, Song J P, Ta D A , Mei Y F, Xu K L 2022 Acta Phys. Sin. 71 174302Google Scholar
[19] Demené C, Robin J, Dizeux A, Heiles B, Pernot M, Tanter M, Perren, Transcranial F 2021 Nat. Biomed. Eng. 5 219Google Scholar
[20] Huang C, Zhang W, Gong P, Lok U W, Chen S 2021 Phys. Med. Biol. 66 8Google Scholar
[21] Couture O, Hingot V, Heiles B, Muleki-Seya P, Tanter M 2018 IEEE Trans. Ultrason. Ferroelectr. Freq. Control 65 1304Google Scholar
[22] Christensen-Jeffries K, Couture O, Dayton P A, Eldar Y, Hynynen K, Kiessling F, O'Reilly M, Pinton G, Schmitz G, Tang M, Tanter M, van Sloun R J G 2020 Ultrasound Med. Biol. 46 4Google Scholar
[23] Youn J, Ommen M L, Stuart M B, Thomsen E V, Jensenet J A 2019 IEEE International Ultrasonics Symposium (IUS)
[24] Sloun R J G v , Solomon O, Bruce M, Khaing Z Z, Wijkstra H, Eldar Y C, Mischi M 2021 IEEE Trans. Med. Imaging 40 829Google Scholar
[25] Liu X, Zhou T, Lu M, Yang Y, He Q, Luo J 2020 IEEE Trans. Med. Imaging 39 3064Google Scholar
[26] Bar-Zion A, Solomon O, Tremblay-Darveau C, Adam D, Eldar Y. C 2018 IEEE Trans. Ultrason. Ferroelectr. Freq. Control 65 2365Google Scholar
[27] Bar-Zion A, Tremblay-Darveau C, Solomon O, Adam D, Eldar Y. C 2016 IEEE Trans. Med. Imaging 36 169Google Scholar
[28] Nieuwenhuizen R P, Lidke K A, Bates M, Puig D L, Grunwald D, Stallinga S, Rieger B 2013 Nat. Methods 10 557Google Scholar
[29] Jensen J A, Holm O, Jerisen L J, Bendsen H, Nikolov I S 2005 IEEE Trans. Ultrason. Ferroelectr. Freq. Control 52 881Google Scholar
[30] Tanter M, Fink M Ultrafast Imaging in Biomedical Ultrasound. 2014 IEEE Trans. Ultrason. Ferroelectr. Freq. Control 61 102Google Scholar
[31] Montaldo G, Tanter M, Bercoff J, Benech N, Fink M 2009 IEEE Trans. Ultrason. Ferroelectr. Freq. Control 56 489Google Scholar
[32] Ledoux L, Brands P J, Hoeks A 1997 Ultrason. Imaging 19 1Google Scholar
[33] Baranger J, Arnal B, Perren F, Baud O, Tanter M, Demené C 2018 IEEE Trans. Med. Imaging 37 1574Google Scholar
[34] Demené C, Deffieux T, Pernot M, Osmanski B F, Biran V, Gennisson J L, Sieu L A, Bergel A, Franqui S, Correas J M 2015 IEEE Trans. Med. Imaging 34 2271Google Scholar
[35] Hingot V, Errico C, Tanter M, Couture O 2017 Ultrasonics 77 17Google Scholar
[36] Heiles B, Chavignon A, Hingot V, Lopez P, Teston E, Couture O 2021 Nat. Biomed. Eng. 6 605
[37] Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y 2014 Adv. Neural Inf. Process. Syst. 27 2672Google Scholar
[38] Ronneberger O, Fischer P, Brox T 2015 Medical Image Computing and Computer-Assisted Intervention, PT III 9351 234
[39] Nitish S, Geoffrey H, Alex K, Ilya S, Ruslan S 2014 J. Mach. Learn. Res. 15 1929Google Scholar
[40] Zhao H, Gallo O, Frosio I, Kautz J 2017 IEEE Trans. Comput. Imaging 3 47Google Scholar
[41] Ouyang W, Aristov A, Hao X, Lelek M, Zimmer C 2018 Nat. Biotechnol. 36 460Google Scholar
[42] Hingot V, Errico C, Heiles B, Rahal L, Tanter M, Couture O 2019 Sci. Rep. 9 2456Google Scholar
[43] Hingot V, Chavignon A, Heiles B, Couture O 2021 IEEE Trans. Med. Imaging 40 3812Google Scholar
-
图 1 超声成像分辨率及微泡的B-mode图像 (a) 两微泡间距恰好为半波长; (b) 两微泡间距在半波长内; (c) 实验测量的微泡点扩散函数及其中心定位(由红点标记)
Figure 1. The resolution of ultrasound imaging and a B-mode image of microbubbles: (a) The two sources are exactly half a wavelength apart; (b) the two sources are within a half-wavelength distance; (c) microbubbles appearing as point spread function and their localizations.
图 2 uULM常规流程 (a) 超快超声数据在体采集; (b) 杂波滤除; (c) 运动校准; (d) 微泡定位; (e) 微泡追踪; (f) 超分辨率图像重建
Figure 2. uULM conventional process: (a) In vivo acquisition of ultrafast ultrasound data ; (b) clutter filtering; (c) motion correction; (d) microbubble localization; (e) microbubble tracking; (f) super-resolution image reconstruction.
图 6 全血管造影的局部特写及其沿白色虚线的强度分布图. 使用标准uULM (a) 和GAN-uULM (b) 分别对体内数据集进行uULM血管造影得到的局部放大图; (c) 绿色和蓝色曲线表示沿水平虚线的强度分布图
Figure 6. Zoomed-in regions of interest from the whole angiogram and their intensity profiles along the white dashed line. Magnified regions from uULM Angiograms for an in-vivo dataset using the standard uULM method (a) and the GAN-uULM (b); (c) the intensity profiles along a given horizontal dashed line overlaid in green and blue.
图 8 不同采集时长对应的超分辨率血流图像 (a), (b) uULM和GAN-uULM在采集时长为40 s时的血流图像; (c), (d) uULM和GAN-uULM在采集时长为80 s时的血流图像
Figure 8. Super-resolution blood flow images with different cumulative acquisition times: (a), (b) The results of uULM and GAN-uULM with acquisition time of 40 s; (c), (d) the results of uULM and GAN-uULM with acquisition time of 80 s.
图 9 基于FRC曲线的分辨率测量 (a), (b) 将重建结果随机拆分的两个子图像; (c), (d) 2D FFT得到频谱图; (e) FRC曲线, FRC曲线与1/2 bit (黄色)阈值曲线的两个交点被用于测定图像分辨率; (f) 分辨率与累积采集时长的关系曲线
Figure 9. Resolution measurements based on FRC curves: (a), (b) Two sub-images obtained by randomly splitting the reconstruction results; (c), (d) the frequency spectrograms obtained by 2D FFT; (e) the FRC curves, the two intersections of the FRC curves with the 1/2 bit (yellow) threshold curve are used to determine the image resolution; (f) the relationship curves between resolution and cumulative acquisition time.
-
[1] 钟传钰, 郑元义 2021 中国医学影像技术 37 1799Google Scholar
Zhong C Y, Zheng Y Y 2021 Chin. J. Med. Imaging Technol. 37 1799Google Scholar
[2] 王宇森, 陶鸿根 1991 中华内分泌代谢杂志 7 2
Wang Y S, Tao H G 1991 Chin. J. Endocrinol. Metab. 7 2
[3] Chugh B P, Lerch J P, Yu L X, Pienkowski M, Harrison R V, Henkelman R M, Sled J G 2009 Neuroimage 47 1312Google Scholar
[4] Huang C H, Chen C C V, Siow T Y, Hsu S H S, Hsu Y H, Jaw F S, Chang C 2013 PLoS One 8 e78186Google Scholar
[5] Hong G, Lee J C, Robinson J T, Raaz U, Xie L M, Huang, N F, Cooke J P, Dai H J 2012 Nat. Med. 18 1841Google Scholar
[6] Yao J, Wang L, Yang J M, Maslov K I, Wong T T W, Li L, Huang C H, Zou J, Wang L V 2015 Nat. Methods 12 407Google Scholar
[7] O"Reilly M A, Hynynen K 2013 Med. Phys. 40 110701Google Scholar
[8] Jiang C, Li Y, Xu K, Ta D 2021 IEEE Trans. Ultrason. Ferroelectr. Freq. Control 68 72Google Scholar
[9] 臧佳琦, 许凯亮, 韩清见, 陆起涌, 梅永丰, 他得安 2021 物理学报 70 114304Google Scholar
Zang J Q, Xu K L, Han Q J, Lu Q Y, Mei Y F, Ta D A 2021 Acta Phys. Sin. 70 114304Google Scholar
[10] Sui Y H, Yan S Y, Zang J Q, Liu X, Ta D A, Wang W Q, Xu K L 2021 IEEE International Ultrasonics Symposium (IUS)
[11] Sui Y H, Yan S Y, Yu J J, Song J P, Ta D A, Wang W Q, Xu K L 2022 IEEE Trans. Ultrason. Ferroelectr. Freq. Control 69 2425Google Scholar
[12] Couture O, Bannouf S, Montaldo G, Aubry J F, Fink M 2009 Ultrasound Med. Biol. 35 1908Google Scholar
[13] Viessmann O M, Eckersley R J, Christensen-Jeffries K, Tang M X, Dunsby C 2013 Phys. Med. Biol. 58 6447Google Scholar
[14] Desailly Y, Couture O, Fink M, Tanter M 2013 Appl. Phys. Lett. 103 189Google Scholar
[15] Errico C, Pierre J, Pezet S, Desailly Y, Lenkei Z, Couture O, Tanter M 2015 Nature 527 499Google Scholar
[16] Fanglue L, Shelton S E, Espíndola D, Rojas J D, Gianmarco P, Dayton P A 2017 Theranostics 7 196Google Scholar
[17] Xu K L, Guo X Y, Sui Y H, Hingot V, Couture O, Ta D A, Wang W Q 2021 IEEE International Ultrasonics Symposium (IUS)
[18] 郁钧瑾, 郭星奕, 隋怡晖, 宋剑平, 他得安, 梅永丰, 许凯亮 2022 物理学报 71 174302Google Scholar
Yu J J, Guo X Y, Sui Y H, Song J P, Ta D A , Mei Y F, Xu K L 2022 Acta Phys. Sin. 71 174302Google Scholar
[19] Demené C, Robin J, Dizeux A, Heiles B, Pernot M, Tanter M, Perren, Transcranial F 2021 Nat. Biomed. Eng. 5 219Google Scholar
[20] Huang C, Zhang W, Gong P, Lok U W, Chen S 2021 Phys. Med. Biol. 66 8Google Scholar
[21] Couture O, Hingot V, Heiles B, Muleki-Seya P, Tanter M 2018 IEEE Trans. Ultrason. Ferroelectr. Freq. Control 65 1304Google Scholar
[22] Christensen-Jeffries K, Couture O, Dayton P A, Eldar Y, Hynynen K, Kiessling F, O'Reilly M, Pinton G, Schmitz G, Tang M, Tanter M, van Sloun R J G 2020 Ultrasound Med. Biol. 46 4Google Scholar
[23] Youn J, Ommen M L, Stuart M B, Thomsen E V, Jensenet J A 2019 IEEE International Ultrasonics Symposium (IUS)
[24] Sloun R J G v , Solomon O, Bruce M, Khaing Z Z, Wijkstra H, Eldar Y C, Mischi M 2021 IEEE Trans. Med. Imaging 40 829Google Scholar
[25] Liu X, Zhou T, Lu M, Yang Y, He Q, Luo J 2020 IEEE Trans. Med. Imaging 39 3064Google Scholar
[26] Bar-Zion A, Solomon O, Tremblay-Darveau C, Adam D, Eldar Y. C 2018 IEEE Trans. Ultrason. Ferroelectr. Freq. Control 65 2365Google Scholar
[27] Bar-Zion A, Tremblay-Darveau C, Solomon O, Adam D, Eldar Y. C 2016 IEEE Trans. Med. Imaging 36 169Google Scholar
[28] Nieuwenhuizen R P, Lidke K A, Bates M, Puig D L, Grunwald D, Stallinga S, Rieger B 2013 Nat. Methods 10 557Google Scholar
[29] Jensen J A, Holm O, Jerisen L J, Bendsen H, Nikolov I S 2005 IEEE Trans. Ultrason. Ferroelectr. Freq. Control 52 881Google Scholar
[30] Tanter M, Fink M Ultrafast Imaging in Biomedical Ultrasound. 2014 IEEE Trans. Ultrason. Ferroelectr. Freq. Control 61 102Google Scholar
[31] Montaldo G, Tanter M, Bercoff J, Benech N, Fink M 2009 IEEE Trans. Ultrason. Ferroelectr. Freq. Control 56 489Google Scholar
[32] Ledoux L, Brands P J, Hoeks A 1997 Ultrason. Imaging 19 1Google Scholar
[33] Baranger J, Arnal B, Perren F, Baud O, Tanter M, Demené C 2018 IEEE Trans. Med. Imaging 37 1574Google Scholar
[34] Demené C, Deffieux T, Pernot M, Osmanski B F, Biran V, Gennisson J L, Sieu L A, Bergel A, Franqui S, Correas J M 2015 IEEE Trans. Med. Imaging 34 2271Google Scholar
[35] Hingot V, Errico C, Tanter M, Couture O 2017 Ultrasonics 77 17Google Scholar
[36] Heiles B, Chavignon A, Hingot V, Lopez P, Teston E, Couture O 2021 Nat. Biomed. Eng. 6 605
[37] Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y 2014 Adv. Neural Inf. Process. Syst. 27 2672Google Scholar
[38] Ronneberger O, Fischer P, Brox T 2015 Medical Image Computing and Computer-Assisted Intervention, PT III 9351 234
[39] Nitish S, Geoffrey H, Alex K, Ilya S, Ruslan S 2014 J. Mach. Learn. Res. 15 1929Google Scholar
[40] Zhao H, Gallo O, Frosio I, Kautz J 2017 IEEE Trans. Comput. Imaging 3 47Google Scholar
[41] Ouyang W, Aristov A, Hao X, Lelek M, Zimmer C 2018 Nat. Biotechnol. 36 460Google Scholar
[42] Hingot V, Errico C, Heiles B, Rahal L, Tanter M, Couture O 2019 Sci. Rep. 9 2456Google Scholar
[43] Hingot V, Chavignon A, Heiles B, Couture O 2021 IEEE Trans. Med. Imaging 40 3812Google Scholar
Catalog
Metrics
- Abstract views: 6358
- PDF Downloads: 198
- Cited By: 0