Advanced Retinex-Net image enhancement method based on value component processing

Zhang Hang-Ying; Wang Xue-Qi; Wang Hua-Ying; Cao Liang-Cai

doi:10.7498/aps.71.20220099

Abstract

When capturing images under low-light lighting conditions, the resulting images often suffer low visibility. Such low-visibility images not only affect the visual effect but also cause many difficulties in practical application. Therefore, image enhancement technology under low-light conditions has always been a challenging problem in image algorithms. Considering that most of the existing image enhancement methods are based on the RGB color space enhancement technology, the correlation among the RGB three primary colors is ignored, which makes the color distortion phenomenon easy to occur when the image is enhanced. To solve the problems of poor image visibility and color deviation under low-light conditions, in this paper an advanced Retinex network enhancement method is proposed. In the method, firstly the low-light RGB image is transformed into HSV color space, the Retinex decomposition network is used to decompose and enhance the value component separately, and thus increasing the resolution of the value component through up-sampling operation; then, for the hue component and saturation component, the nearest neighbor interpolation is used to increase their resolutions, and the enhanced value component is combined to convert back to RGB color space to obtain the initial enhanced image; finally, the wavelet transform image fusion technology is used to fuse with the original low-light image to eliminate the over-enhanced part in the initial enhanced image. The analysis of experimental results shows that the method proposed in this paper has obvious advantages in brightness enhancement and color restoration of low-light images. Especially, comparing with the original Retinex network method, the NIQE value decreases by an average of 19.49%, and the image standard deviation increases by an average of 41.35%. The algorithm proposed in this paper is expected to be effectively used in the fields of security monitoring and biomedicine.

Keywords:

Authors and contacts

1.
State Key Laboratory of Precision Testing Technology and Instruments, Department of Precision Instruments, Tsinghua University, Beijing 100084, China
2.
School of Mathematical Science and Engineering, Hebei University of Engineering, Handan 056038, China

Corresponding author: Cao Liang-Cai, clc@tsinghua.edu.cn

Funds: Project supported by the National Natural Science Foundation of China (Grant No. 61827825 ).

FullText HTML : translate this paragraph

References

[1]	蒋一纯, 詹伟达, 朱德鹏 2021 激光与光电子学进展 58 0410001 Google Scholar Jiang Y C, Zhan W D, Zhu D P 2021 Laser Optoelectron. Prog. 58 0410001 Google Scholar
[2]	韩平丽, 刘飞, 张广, 陶禹, 邵晓鹏 2018 物理学报 67 054202 Google Scholar Han P L, Liu F, Zhang G, Tao Y, Shao X P 2018 Acta Phys. Sin. 67 054202 Google Scholar
[3]	Liu J, Wang X, Chen M, Liu S G, Zhou X R, Shao Z F, Liu P 2014 Opt. Express 22 618 Google Scholar
[4]	Fu X Y, Zeng D L, Huang Y, Liao Y H, Ding X H, Paisley J 2016 Signal Process. 129 82 Google Scholar
[5]	Singh N, Bhandari A K 2021 IEEE Trans. Instrum. Meas. 70 1
[6]	Land E H 1964 Am. Sci. 52 247
[7]	Land E H, McCann J J 1971 J. Opt. Soc. Am. 61 1 Google Scholar
[8]	Land E H, Hubel D H, Livingstone M S, Perry S H, Burns M M 1983 Nature 303 616 Google Scholar
[9]	李红, 吴炜, 杨晓敏, 严斌宇, 刘凯, Gwanggil J 2016 物理学报 65 160701 Google Scholar Li H, Wu W, Yang X M, Yan B Y, Liu K, Gwanggil J 2016 Acta Phys. Sin. 65 160701 Google Scholar
[10]	Jobson D J, Rahman Z, Woodell G A 1997 IEEE Trans. Image Process. 6 451 Google Scholar
[11]	Rahman Z, Jobson D J, Woodell G A 1996 Proceedings of 3rd IEEE International Conference on Image Processing Lausanne, Switzerland, September 19, 1996 p1003
[12]	Jobson D J, Rahman Z, Woodell G A 1997 IEEE Trans. Image Process. 6 965 Google Scholar
[13]	毕国玲, 续志军, 赵建, 孙强 2015 物理学报 64 100701 Google Scholar Bi G L, Xu Z J, Zhao J, Sun Q 2015 Acta Phys. Sin. 64 100701 Google Scholar
[14]	Zhou Z Q, Dong M J, Xie X Z, Gao Z F 2016 Appl. Opt. 55 6480
[15]	王殿伟, 韩鹏飞, 范九伦, 刘颖, 许志杰, 王晶 2018 物理学报 67 210701 Google Scholar Wang D W, Han P F, Fan J L, Liu Y, Xu Z J, Wang J 2018 Acta Phys. Sin. 67 210701 Google Scholar
[16]	Kwon H J, Lee S H, Lee G Y, Sohng K I 2014 Digit. Signal Process. 30 74 Google Scholar
[17]	Yang Q X, Tan K H, Ahuja N 2009 IEEE Conference on Computer Vision and Pattern Recognition Miami, USA, June 20–25, 2009 p557
[18]	Wang S H, Zheng J, Hu H M, Li B 2013 IEEE Trans. Image Process. 22 3538 Google Scholar
[19]	Fu X Y, Zeng D L, Huang Y, Zhang X P, Ding X H 2016 IEEE Conference on Computer Vision and Pattern Recognition Las Vegas, USA, June 27–30, 2016 p2782
[20]	Guo X J, Li Y, Ling H B 2017 IEEE Trans. Image Process. 26 982 Google Scholar
[21]	Gijsenij A, Gevers T, Weijer J 2011 IEEE Trans. Image Process. 20 2475 Google Scholar
[22]	赵欣慰, 金韬, 池灏, 曲嵩 2015 物理学报 64 104201 Google Scholar Zhao X W, Jin T, Chi H, Qu S 2015 Acta Phys. Sin. 64 104201 Google Scholar
[23]	Jiang Z Q, Li H T, Liu L j, Men A D, Wang H Y 2021 Neurocomputing 454 361 Google Scholar
[24]	马红强, 马时平, 许悦雷, 朱明明 2019 光学学报 39 0210004 Google Scholar Ma H Q, Ma S P, Xu Y L, Zhu M M 2019 Acta Opt. Sin. 39 0210004 Google Scholar
[25]	Guo Y H, Ke X, Ma J, Zhang J 2019 IEEE Access 7 13737 Google Scholar
[26]	Lore K G, Akintayo A, Sarkar S 2017 Pattern Recognit. 61 650 Google Scholar
[27]	Wang W J, Wei C, Yang W H, Liu J Y 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition Xi'an, China, May 15–19, 2018 p751
[28]	He W J, Liu Y Y, Feng J F, Zhang W W, Gu G H, Chen Q 2020 IEEE 3rd International Conference on Information Systems and Computer Aided Education Dalian, China, September 27–29, 2020 p397
[29]	Wei C, Wang W J, Yang W H, Liu J Y 2018 arXiv: 1808.04560 v1 [cs. CV]
[30]	Yakno M, Mohamad-Saleh J, Ibrahim M Z 2021 Sensors 21 6445 Google Scholar
[31]	陈刚, 刘言, 杨贺超, 孙斌, 喻春雨 2021 光学精密工程 29 1999 Google Scholar Chen G, Liu Y, Yang H C, Sun B, Yu C Y 2021 Opt. Precis. Eng. 29 1999 Google Scholar
[32]	Zhang H Y, Cao L C, Yang F 2021 Proc. SPIE First Optics Frontier Conference Hangzhou, China, June 18, 2021 1185002
[33]	Yadav A K, Roy R, Kumar A P, Kumar C S, Dhakad S K 2015 International Conference on Advances in Computing, Communications and Informatics Kochi, India, August 10–13, 2015 p1204
[34]	Mittal A, Soundararajan R, Bovik A C 2013 IEEE Signal Process. Lett. 20 209 Google Scholar

Cited By

图 1 (a) Retinex成像理论模型; (b) Retinex算术思想简介

Figure 1. (a) Retinex imaging theoretical model; (b) arithmetic ideas of Retinex algorithm.

DownLoad: Full-Size Img PowerPoint

图 2 改进的Retinex网络增强算法流程图

Figure 2. Flow chart of the advanced Retinex network enhancement algorithm.

DownLoad: Full-Size Img PowerPoint

图 3 sym4尺度函数与小波函数

Figure 3. Scale function and wavelet function of sym4.

DownLoad: Full-Size Img PowerPoint

图 4 不同算法对比效果图

Figure 4. Comparison of different algorithms.

DownLoad: Full-Size Img PowerPoint

图 5 不同算法增强效果局部放大图

Figure 5. Local enlarged view of the enhancement effect of different algorithms.

DownLoad: Full-Size Img PowerPoint

图 6 不同方法图像评价指标的均值变化情况

Figure 6. Changes in mean values of image evaluation metrics for different methods.

DownLoad: Full-Size Img PowerPoint

表 1 V分量分解网络结构

Table 1. V-component decomposition network structure.

输入	操作	卷积核	输出通道	步长	输出
RGB	rgb to hsv	—	—	—	H, S, V
V	conv	3	64	1	feats0
feats0	conv & ReLU	3	64	1	feats1
feats1	conv & ReLU	3	64	1	feats2
feats2	conv & ReLU	3	64	1	feats3
feats3	conv & ReLU	3	64	1	feats4
feats4	conv & ReLU	3	64	1	feats5
feats5	conv & sigmoid	3	2	1	R, I

DownLoad: CSV

表 2 V分量增强网络结构

Table 2. V-component enhancement network structure.

输入	操作	卷积核	输出通道	步长	输出
V_low, R_low, I_low	up-sample	—	—	—	Input
Input	conv	3	64	1	out0
out0	conv & ReLU	3	64	2	out1
out1	conv & ReLU	3	64	2	out2
out2	conv & ReLU	3	64	2	out3
out3	interpolation	—	64	—	out3 up
out3 up, out2 de1	conv & ReLU interpolation	3 —	64 64	1 —	de1 de1 up
de1 up, out1 de2	conv & ReLU interpolation	3 —	64 64	1 —	de2 de2 up
de2 up, out0de1 de2	conv & ReLUinterpolation interpolation	3— —	6464 64	1— —	de3de1 r de2 r
de1 r, de2 r, de3	conv & ReLU	3	64	1	feats0
feats0	conv	1	64	1	feats1
feats1	conv	3	1	1	V_new

DownLoad: CSV

表 3 不同图像的客观评价指标

Table 3. Objective evaluation metrics for different images.

Image	Evaluate	MSRCR	Auto GC	Retinex-Net	ARN
Image1	NIQE	5.6692	5.1384	5.9782	4.0729
	Entropy	7.1095	6.6392	7.1375	7.8179
	SD	33.3758	41.6959	31.1130	42.9601
Image 2	NIQE	6.2926	6.0252	5.3596	3.7336
	Entropy	7.3012	7.5171	7.5777	7.7226
	SD	41.8903	55.6071	46.3428	66.2911
Image 3	NIQE	5.6715	4.9203	4.4528	4.0319
	Entropy	6.7898	7.1224	7.7284	7.8633
	SD	31.3800	37.3028	53.5654	72.4424
Image 4	NIQE	3.7695	3.8844	3.7200	3.6582
	Entropy	5.5392	7.1881	7.2807	7.4010
	SD	40.8917	38.6741	39.9913	46.9674
Image 5	NIQE	3.9541	4.4738	4.0126	3.6424
	Entropy	7.3497	6.0549	7.2871	7.4387
	SD	42.1574	41.0863	32.5321	56.5474
Image 6	NIQE	7.3401	6.4273	5.4459	3.8790
	Entropy	7.0335	5.5701	7.3417	7.8134
	SD	34.1136	40.6108	38.4974	56.5800
Mean	NIQE	5.4495	5.1449	4.7649	3.8363
	Entropy	6.8538	6.6820	7.3922	7.6762
	SD	37.3015	42.4962	40.3003	56.9647

DownLoad: CSV

[1]	蒋一纯, 詹伟达, 朱德鹏 2021 激光与光电子学进展 58 0410001 Google Scholar Jiang Y C, Zhan W D, Zhu D P 2021 Laser Optoelectron. Prog. 58 0410001 Google Scholar
[2]	韩平丽, 刘飞, 张广, 陶禹, 邵晓鹏 2018 物理学报 67 054202 Google Scholar Han P L, Liu F, Zhang G, Tao Y, Shao X P 2018 Acta Phys. Sin. 67 054202 Google Scholar
[3]	Liu J, Wang X, Chen M, Liu S G, Zhou X R, Shao Z F, Liu P 2014 Opt. Express 22 618 Google Scholar
[4]	Fu X Y, Zeng D L, Huang Y, Liao Y H, Ding X H, Paisley J 2016 Signal Process. 129 82 Google Scholar
[5]	Singh N, Bhandari A K 2021 IEEE Trans. Instrum. Meas. 70 1
[6]	Land E H 1964 Am. Sci. 52 247
[7]	Land E H, McCann J J 1971 J. Opt. Soc. Am. 61 1 Google Scholar
[8]	Land E H, Hubel D H, Livingstone M S, Perry S H, Burns M M 1983 Nature 303 616 Google Scholar
[9]	李红, 吴炜, 杨晓敏, 严斌宇, 刘凯, Gwanggil J 2016 物理学报 65 160701 Google Scholar Li H, Wu W, Yang X M, Yan B Y, Liu K, Gwanggil J 2016 Acta Phys. Sin. 65 160701 Google Scholar
[10]	Jobson D J, Rahman Z, Woodell G A 1997 IEEE Trans. Image Process. 6 451 Google Scholar
[11]	Rahman Z, Jobson D J, Woodell G A 1996 Proceedings of 3rd IEEE International Conference on Image Processing Lausanne, Switzerland, September 19, 1996 p1003
[12]	Jobson D J, Rahman Z, Woodell G A 1997 IEEE Trans. Image Process. 6 965 Google Scholar
[13]	毕国玲, 续志军, 赵建, 孙强 2015 物理学报 64 100701 Google Scholar Bi G L, Xu Z J, Zhao J, Sun Q 2015 Acta Phys. Sin. 64 100701 Google Scholar
[14]	Zhou Z Q, Dong M J, Xie X Z, Gao Z F 2016 Appl. Opt. 55 6480
[15]	王殿伟, 韩鹏飞, 范九伦, 刘颖, 许志杰, 王晶 2018 物理学报 67 210701 Google Scholar Wang D W, Han P F, Fan J L, Liu Y, Xu Z J, Wang J 2018 Acta Phys. Sin. 67 210701 Google Scholar
[16]	Kwon H J, Lee S H, Lee G Y, Sohng K I 2014 Digit. Signal Process. 30 74 Google Scholar
[17]	Yang Q X, Tan K H, Ahuja N 2009 IEEE Conference on Computer Vision and Pattern Recognition Miami, USA, June 20–25, 2009 p557
[18]	Wang S H, Zheng J, Hu H M, Li B 2013 IEEE Trans. Image Process. 22 3538 Google Scholar
[19]	Fu X Y, Zeng D L, Huang Y, Zhang X P, Ding X H 2016 IEEE Conference on Computer Vision and Pattern Recognition Las Vegas, USA, June 27–30, 2016 p2782
[20]	Guo X J, Li Y, Ling H B 2017 IEEE Trans. Image Process. 26 982 Google Scholar
[21]	Gijsenij A, Gevers T, Weijer J 2011 IEEE Trans. Image Process. 20 2475 Google Scholar
[22]	赵欣慰, 金韬, 池灏, 曲嵩 2015 物理学报 64 104201 Google Scholar Zhao X W, Jin T, Chi H, Qu S 2015 Acta Phys. Sin. 64 104201 Google Scholar
[23]	Jiang Z Q, Li H T, Liu L j, Men A D, Wang H Y 2021 Neurocomputing 454 361 Google Scholar
[24]	马红强, 马时平, 许悦雷, 朱明明 2019 光学学报 39 0210004 Google Scholar Ma H Q, Ma S P, Xu Y L, Zhu M M 2019 Acta Opt. Sin. 39 0210004 Google Scholar
[25]	Guo Y H, Ke X, Ma J, Zhang J 2019 IEEE Access 7 13737 Google Scholar
[26]	Lore K G, Akintayo A, Sarkar S 2017 Pattern Recognit. 61 650 Google Scholar
[27]	Wang W J, Wei C, Yang W H, Liu J Y 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition Xi'an, China, May 15–19, 2018 p751
[28]	He W J, Liu Y Y, Feng J F, Zhang W W, Gu G H, Chen Q 2020 IEEE 3rd International Conference on Information Systems and Computer Aided Education Dalian, China, September 27–29, 2020 p397
[29]	Wei C, Wang W J, Yang W H, Liu J Y 2018 arXiv: 1808.04560 v1 [cs. CV]
[30]	Yakno M, Mohamad-Saleh J, Ibrahim M Z 2021 Sensors 21 6445 Google Scholar
[31]	陈刚, 刘言, 杨贺超, 孙斌, 喻春雨 2021 光学精密工程 29 1999 Google Scholar Chen G, Liu Y, Yang H C, Sun B, Yu C Y 2021 Opt. Precis. Eng. 29 1999 Google Scholar
[32]	Zhang H Y, Cao L C, Yang F 2021 Proc. SPIE First Optics Frontier Conference Hangzhou, China, June 18, 2021 1185002
[33]	Yadav A K, Roy R, Kumar A P, Kumar C S, Dhakad S K 2015 International Conference on Advances in Computing, Communications and Informatics Kochi, India, August 10–13, 2015 p1204
[34]	Mittal A, Soundararajan R, Bovik A C 2013 IEEE Signal Process. Lett. 20 209 Google Scholar

[1]	Shi Yue, Ou Pan, Zheng Ming, Tai Han-Xu, Wang Yu-Hong, Duan Ruo-Nan, Wu Jian. Artifact noise suppression of particle-field computed tomography based on lightweight residual and enhanced convergence neural network. Acta Physica Sinica, 2024, 73(10): 104202. doi: 10.7498/aps.73.20231902
[2]	Liu Hong-Jiang, Liu Yi-Fei, Gu Fu-Xing. Automatic fabrication system of optical micro-nanofiber based on deep learning. Acta Physica Sinica, 2024, 73(10): 104207. doi: 10.7498/aps.73.20240171
[3]	Zhan Qing-Liang, Ge Yao-Jun, Bai Chun-Jin. Flow feature extraction models based on deep learning. Acta Physica Sinica, 2022, 71(7): 074701. doi: 10.7498/aps.71.20211373
[4]	Zhang Yao, Zhang Yun-Bo, Chen Li. Deep-learning-assisted micro impurity detection on an optical surface. Acta Physica Sinica, 2021, 70(16): 168702. doi: 10.7498/aps.70.20210403
[5]	Wang Tian-Tian, Wang Hui, Zhu Yan-Chun, Wang Li-Jia. Motion tracking of left myocardium in cardiac cine magnetic resonance image based on displacement flow U-Net and variational autoencoder. Acta Physica Sinica, 2021, 70(22): 228701. doi: 10.7498/aps.70.20210885
[6]	Nan Hu, Ma Xiao-Jing, Zhao Hai-Bo, Tang Shao-Jie, Liu Wei-Hua, Wang Da-Wei, Jia Chun-Lin. Detection of intensity peaks in high-resolution transmission electron microscopy image based on YOLOv3. Acta Physica Sinica, 2021, 70(7): 076803. doi: 10.7498/aps.70.20201502
[7]	Su Bo, Tao Fen, Li Ke, Du Guo-Hao, Zhang Ling, Li Zhong-Liang, Deng Biao, Xie Hong-Lan, Xiao Ti-Qiao. Image alignment for synchrotron radiation based X-ray nano-CT. Acta Physica Sinica, 2021, 70(16): 160704. doi: 10.7498/aps.70.20210156
[8]	Zhou Jing, Zhang Xiao-Fang, Zhao Yan-Geng. Phase retrieval wavefront sensing based on image fusion and convolutional neural network. Acta Physica Sinica, 2021, 70(5): 054201. doi: 10.7498/aps.70.20201362
[9]	Zhao Zhi-Peng, Zhou Shuang, Wang Xing-Yuan. A new chaotic signal based on deep learning and its application in image encryption. Acta Physica Sinica, 2021, 70(23): 230502. doi: 10.7498/aps.70.20210561
[10]	Lang Li-Ying, Lu Jia-Lei, Yu Na-Na, Xi Si-Xing, Wang Xue-Guang, Zhang Lei, Jiao Xiao-Xue. In depth learning based method of denoising joint transform correlator optical image encryption system. Acta Physica Sinica, 2020, 69(24): 244204. doi: 10.7498/aps.69.20200805
[11]	Chen Wei, Guo Yuan, Jing Shi-Wei. General image encryption algorithm based on deep learning compressed sensing and compound chaotic system. Acta Physica Sinica, 2020, 69(24): 240502. doi: 10.7498/aps.69.20201019
[12]	Liu Jie, Zhang Jian-Xun, Dai Yu. Image enhancement based on multi-guided filtering. Acta Physica Sinica, 2018, 67(23): 238701. doi: 10.7498/aps.67.20181425
[13]	Liu Qi, Wang Li-Dan, Duan Shu-Kai. An adaptive three-Gauss model based on memristive cross array and its application in image enhancement. Acta Physica Sinica, 2017, 66(12): 127301. doi: 10.7498/aps.66.127301
[14]	Wang Cong, Yang Jing, Pan Xiu-Juan, Cai Gao-Hang, Zhao Wei, Zhang Jing-Yuan, Cui Da-Fu, Peng Qin-Jun, Xu Zu-Yan. Image restoration and enhancement based on phase conjugation of optical parametric amplification. Acta Physica Sinica, 2017, 66(10): 104205. doi: 10.7498/aps.66.104205
[15]	Li Hong, Wu Wei, Yang Xiao-Min, Yan Bin-Yu, Liu Kai, Gwanggil Jeon. Multispectral image enhancement based on Retinex by using structure extraction. Acta Physica Sinica, 2016, 65(16): 160701. doi: 10.7498/aps.65.160701
[16]	Bi Guo-Ling, Xu Zhi-Jun, Zhao Jian, Sun Qiang. Multispectral image enhancement based on irradiation-reflection model and bounded operation. Acta Physica Sinica, 2015, 64(10): 100701. doi: 10.7498/aps.64.100701
[17]	Feng Xin, Li Chuan, Hu Kai-Qun. Infrared and visible image fusion based on deep Boltzmann model. Acta Physica Sinica, 2014, 63(18): 184202. doi: 10.7498/aps.63.184202
[18]	Zhao Wen-Da, Zhao Jian, Xu Zhi-Jun. Variational multi-source image fusion based on the structure tensor. Acta Physica Sinica, 2013, 62(21): 214204. doi: 10.7498/aps.62.214204
[19]	Gan Tian, Feng Shao-Tong, Nie Shou-Ping, Zhu Zhu-Qing. Image fusion algorithm based on block-DCT in wavelet domain. Acta Physica Sinica, 2011, 60(11): 114205. doi: 10.7498/aps.60.114205
[20]	Zhang Chuang, Bai Lian-Fa, Zhang Yi. Method of fusing dual-spectrum low light Level images based on gray-scale spatial correlation. Acta Physica Sinica, 2007, 56(6): 3227-3233. doi: 10.7498/aps.56.3227

Catalog

Vol. 71, Issue 11 - 2022-06-05

Metrics

Abstract views: 6445
PDF Downloads: 144
Cited By: 0

姓名
邮箱
手机号码
标题
留言内容
验证码

Search

Article

留言板