搜索

x

留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

基于生物视觉特征和视觉心理学的视频显著性检测算法

方志明 崔荣一 金璟璇

引用本文:
Citation:

基于生物视觉特征和视觉心理学的视频显著性检测算法

方志明, 崔荣一, 金璟璇

Video saliency detection algorithm based on biological visual feature and visual psychology theory

Fang Zhi-Ming, Cui Rong-Yi, Jin Jing-Xuan
PDF
导出引用
  • 提出了一种空域和时域相结合的视频显著性检测算法.对单帧图像,受视觉皮层层次化感知特性和Gestalt视觉心理学的启发,提出了一种层次化的静态显著图检测方法.在底层,通过符合生物视觉特性的特征图像(双对立颜色特征及亮度特征图像)的非线性简化模型来合成特征图像,形成多个候选显著区域;在中层,根据矩阵的最小Frobenius-范数(F-范数)性质选取竞争力最强的候选显著区域作为局部显著区域;在高层,利用Gestalt视觉心理学的核心理论,对在中层得到的局部显著区域进行整合,得到具有整体感知的空域显著图.对序列帧图像,基于运动目标在位置、运动幅度和运动方向一致性的假设,对Lucas-Kanade算法检测出的光流点进行二分类,排除噪声点的干扰,并利用光流点的运动幅度来衡量运动目标运动显著性.最后,基于人类视觉对动态信息与静态信息敏感度的差异提出了一种空域和时域显著图融合的通用模型.实验结果表明,该方法能够抑制视频背景中的噪声并且解决了运动目标稀疏等问题,能够较好地从复杂场景中检测出视频中的显著区域.
    In order to solve the problems of video saliency detection and poor fusion effect, a video saliency detection model and a fusion model are proposed. Video saliency detection is divided into spatial saliency detection and temporal saliency detection. In the spatial domain, inspired by the properties of visual cortex hierarchical perception and the Gestalt visual psychology, we propose a hierarchical saliency detection model with three-layer architecture for single frame image. The video single frame is simplified layer by layer, then the results are combined to form a whole consciousness vision object and become easier to deal with. At the bottom of the model, candidate saliency regions are formed by nonlinear simplification model of the characteristic image (dual color characteristic and luminance characteristic image), which is in accordance with the biological visual characteristic. In the middle of the model, the candidate regions with the strongest competitiveness are selected as the local salient regions according to the property of matrix minimum Fresenius- norm (F- norm). At the top level of the model, the local salient regions are integrated by the core theory of Gestalt visual psychology, and the spatial saliency map is obtained. In the time domain, based on the consistency assumption of a moving object in target location, motion range and direction, the optical flow points detected by Lucas-Kanade method are classified to eliminate the noise interference, then the motion saliency of moving object is measured by the motion amplitude. Finally, based on the difference between the visual sensitivity of dynamic and static information and the difference in visual sensitivity between color information and gray information, a general fusion model of time and spatial domain salient region is proposed. The saliency detection results of single frame image and video sequence frame image are represented by the gray color model and the Munsell color system respectively. Experimental results show that the proposed saliency detection method can suppress the background noise, solve the sparse pixels problem of a moving object, and can effectively detect the salient regions from the video. The proposed fusion model can display two kinds of saliency results simultaneously in a single picture of a complex scene. This model ensures that the detection results of images are so complicated that a chaotic situation will not appear.
      通信作者: 金璟璇, 1537161104@qq.com
    • 基金项目: 吉林省科技发展计划项目(批准号:20140101186JC)资助的课题.
      Corresponding author: Jin Jing-Xuan, 1537161104@qq.com
    • Funds: Project supported by the Science and Technology Development Plan Foundation of Jilin Province, China (Grant No. 20140101186JC).
    [1]

    Borji A, Sihite D N, Itti L 2015 IEEE Trans. Image Process. 24 5706

    [2]

    Cichy R M, Pantazis D, Oliva A 2016 Cerebral Cortex 26 3563

    [3]

    Li Z C, Qin S Y, Itti L 2011 Image Vision Comput. 29 1

    [4]

    Wu G L, Fu Y J, Huang S C, Chen S Y 2013 IEEE Trans. Image Process. 22 2247

    [5]

    Franke U, Pfeiffer D, Rabe C, Knoeppel C, Enzweiler M, Stein F, Herrtwich R 2013 Proceedings of IEEE Conference on Computer Vision Sydney, Australia, December 1-8, 2013 p214

    [6]

    Ma Y F, Hua X S, Lu L, Zhang H J 2005 IEEE Trans. Multimed. 7 907

    [7]

    Ejaz N, Mehmood I, Baik S W 2014 Comput. Elec. Engr. 40 993

    [8]

    Evangelopoulos G, Zlatintsi A, Potamianos A, Maragos P 2013 IEEE Trans. Multimed. 15 1553

    [9]

    Itti L, Koch C, Niebur E 1998 IEEE Trans. Pattern Anal. Mach. Intell. 20 1254

    [10]

    Itti L, Koch C 2001 Nat. Rev. Neurosci. 2 194

    [11]

    Cheng M M, Zhang G X, Mitra N J, Huang X, Hu S M 2011 Proceedings of Computer Vision and Pattern Recognition Colorado Springs, November 15-18, 2011 p409

    [12]

    Liu J, Wang S 2015 Neurocomputing 147 435

    [13]

    Guo C, Ma Q, Zhang L 2008 Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition Anchorage, Alaska, January 17-18, 2008 p1

    [14]

    Hou X D, Zhang L Q 2007 Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition Minneapolis, Minnesota, June 19-21, 2007 p18

    [15]

    Zhu Z, Wang M 2016 J. Comput. Appl. 36 2560

    [16]

    Tao D, Cheng J, Song M 2016 IEEE Trans. Neur. Netw. Lear. Syst. 27 1122

    [17]

    Xue Y W, Guo X J, Cao X C 2012 Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing Kyoto, Japan, March 25-30, 2012 p1485

    [18]

    Ma Z M, Tao C K 1999 Acta Phys. Sin. 48 2202 (in Chinese) [马兆勉, 陶纯堪 1999 物理学报 48 2202]

    [19]

    Jin Z L, Han J, Zhang Y, Bo L F 2014 Acta Phys. Sin. 63 069501 (in Chinese) [金左轮, 韩静, 张毅, 柏连发 2014 物理学报 63 069501]

    [20]

    Wu Y Q, Zhang J K 2010 Acta Phys. Sin. 59 5487 (in Chinese) [吴一全, 张金矿 2010 物理学报 59 5487]

    [21]

    Xu Y N, Zhao Y, Liu L P, Zhang Y, Sun X D 2010 Acta Phys. Sin. 59 980 (in Chinese) [许元男, 赵远, 刘丽萍, 张宇, 孙秀冬 2010 物理学报 59 980]

    [22]

    Wang X, Ma H, Chen X 2016 Proceedings of International Conference on Image Processing the Phoenix Convention Centre, Phoenix, Arizona, USA, September, 2016 p25

    [23]

    He S, Lau R W, Liu W 2015 Int. J. Comput. Vision 115 330

    [24]

    Li H, Chen J, Lu H 2017 Neurocomputing 226 212

    [25]

    Huang Y 2016 M. S. Thesis (Beijing: Institute of Optoelectronic Technology) [黄烨2016 硕士学位论文 (北京: 中国科学院)]

    [26]

    Paragios N, Deriche R 2000 IEEE Trans. Pattern Anal. Mach. Intell. 22 266

    [27]

    Tsai D M, Lai S C 2009 IEEE Trans. Image Process. 18 158

    [28]

    Barron J L, Fleet D, Beauchemin S S 1994 Int. J. Comput. Vision 12 43

    [29]

    Elazary L, Itti L 2008 J. Vision 8

    [30]

    Lucas B D, Kanade T 1981 Proceedings of International Joint Conference on Artificial Intelligence Vancouver, BC, Canada, August, 1981 285

    [31]

    Baker S, Scharstein D, Lewis J P, Roth S, Black M J, Szelisk R 2007 Proceedings of IEEE International Conference on Computer Vision Rio de Janeiro, Brazil, October 14-21, 2007 p92

    [32]

    Koffka K 1935 Principles of Gestalt Psychology (London: Lund Humphries)

    [33]

    Mullen K T 1985 J. Phys. 359 381

    [34]

    Gary B, Adrian K 2008 Learning OpenCV (America: O'Reilly Media) pp356-370

    [35]

    Shi J, Yan Q, Xu L, Jia J 2016 IEEE Trans. Pattern Anal. Mach. Intell. 38 1

    [36]

    Li X, Li Y, Shen C H, Dick A, Hengel 2013 Proceedings of Computer Vision Sydney, NSW, Australia, December 8, 2013 p3328

  • [1]

    Borji A, Sihite D N, Itti L 2015 IEEE Trans. Image Process. 24 5706

    [2]

    Cichy R M, Pantazis D, Oliva A 2016 Cerebral Cortex 26 3563

    [3]

    Li Z C, Qin S Y, Itti L 2011 Image Vision Comput. 29 1

    [4]

    Wu G L, Fu Y J, Huang S C, Chen S Y 2013 IEEE Trans. Image Process. 22 2247

    [5]

    Franke U, Pfeiffer D, Rabe C, Knoeppel C, Enzweiler M, Stein F, Herrtwich R 2013 Proceedings of IEEE Conference on Computer Vision Sydney, Australia, December 1-8, 2013 p214

    [6]

    Ma Y F, Hua X S, Lu L, Zhang H J 2005 IEEE Trans. Multimed. 7 907

    [7]

    Ejaz N, Mehmood I, Baik S W 2014 Comput. Elec. Engr. 40 993

    [8]

    Evangelopoulos G, Zlatintsi A, Potamianos A, Maragos P 2013 IEEE Trans. Multimed. 15 1553

    [9]

    Itti L, Koch C, Niebur E 1998 IEEE Trans. Pattern Anal. Mach. Intell. 20 1254

    [10]

    Itti L, Koch C 2001 Nat. Rev. Neurosci. 2 194

    [11]

    Cheng M M, Zhang G X, Mitra N J, Huang X, Hu S M 2011 Proceedings of Computer Vision and Pattern Recognition Colorado Springs, November 15-18, 2011 p409

    [12]

    Liu J, Wang S 2015 Neurocomputing 147 435

    [13]

    Guo C, Ma Q, Zhang L 2008 Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition Anchorage, Alaska, January 17-18, 2008 p1

    [14]

    Hou X D, Zhang L Q 2007 Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition Minneapolis, Minnesota, June 19-21, 2007 p18

    [15]

    Zhu Z, Wang M 2016 J. Comput. Appl. 36 2560

    [16]

    Tao D, Cheng J, Song M 2016 IEEE Trans. Neur. Netw. Lear. Syst. 27 1122

    [17]

    Xue Y W, Guo X J, Cao X C 2012 Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing Kyoto, Japan, March 25-30, 2012 p1485

    [18]

    Ma Z M, Tao C K 1999 Acta Phys. Sin. 48 2202 (in Chinese) [马兆勉, 陶纯堪 1999 物理学报 48 2202]

    [19]

    Jin Z L, Han J, Zhang Y, Bo L F 2014 Acta Phys. Sin. 63 069501 (in Chinese) [金左轮, 韩静, 张毅, 柏连发 2014 物理学报 63 069501]

    [20]

    Wu Y Q, Zhang J K 2010 Acta Phys. Sin. 59 5487 (in Chinese) [吴一全, 张金矿 2010 物理学报 59 5487]

    [21]

    Xu Y N, Zhao Y, Liu L P, Zhang Y, Sun X D 2010 Acta Phys. Sin. 59 980 (in Chinese) [许元男, 赵远, 刘丽萍, 张宇, 孙秀冬 2010 物理学报 59 980]

    [22]

    Wang X, Ma H, Chen X 2016 Proceedings of International Conference on Image Processing the Phoenix Convention Centre, Phoenix, Arizona, USA, September, 2016 p25

    [23]

    He S, Lau R W, Liu W 2015 Int. J. Comput. Vision 115 330

    [24]

    Li H, Chen J, Lu H 2017 Neurocomputing 226 212

    [25]

    Huang Y 2016 M. S. Thesis (Beijing: Institute of Optoelectronic Technology) [黄烨2016 硕士学位论文 (北京: 中国科学院)]

    [26]

    Paragios N, Deriche R 2000 IEEE Trans. Pattern Anal. Mach. Intell. 22 266

    [27]

    Tsai D M, Lai S C 2009 IEEE Trans. Image Process. 18 158

    [28]

    Barron J L, Fleet D, Beauchemin S S 1994 Int. J. Comput. Vision 12 43

    [29]

    Elazary L, Itti L 2008 J. Vision 8

    [30]

    Lucas B D, Kanade T 1981 Proceedings of International Joint Conference on Artificial Intelligence Vancouver, BC, Canada, August, 1981 285

    [31]

    Baker S, Scharstein D, Lewis J P, Roth S, Black M J, Szelisk R 2007 Proceedings of IEEE International Conference on Computer Vision Rio de Janeiro, Brazil, October 14-21, 2007 p92

    [32]

    Koffka K 1935 Principles of Gestalt Psychology (London: Lund Humphries)

    [33]

    Mullen K T 1985 J. Phys. 359 381

    [34]

    Gary B, Adrian K 2008 Learning OpenCV (America: O'Reilly Media) pp356-370

    [35]

    Shi J, Yan Q, Xu L, Jia J 2016 IEEE Trans. Pattern Anal. Mach. Intell. 38 1

    [36]

    Li X, Li Y, Shen C H, Dick A, Hengel 2013 Proceedings of Computer Vision Sydney, NSW, Australia, December 8, 2013 p3328

  • [1] 龙康, 孔明, 刘璐, 刘维, 华杭波, 许新科. 基于Lomb-Scargle算法的激光扫频干涉非线性校正方法. 物理学报, 2023, 72(3): 034205. doi: 10.7498/aps.72.20221754
    [2] 华洪涛, 陆博, 古华光. 兴奋性自突触引起神经簇放电频率降低或增加的非线性机制. 物理学报, 2020, 69(9): 090502. doi: 10.7498/aps.69.20191709
    [3] 蒋涛, 黄金晶, 陆林广, 任金莲. 非线性薛定谔方程的高阶分裂改进光滑粒子动力学算法. 物理学报, 2019, 68(9): 090203. doi: 10.7498/aps.68.20190169
    [4] 缪培贤, 杨世宇, 王剑祥, 廉吉庆, 涂建辉, 杨炜, 崔敬忠. 抽运-检测型非线性磁光旋转铷原子磁力仪的研究. 物理学报, 2017, 66(16): 160701. doi: 10.7498/aps.66.160701
    [5] 刘群群, 何文平, 顾斌. 非线性动力学方法在气候突变检测中的应用. 物理学报, 2015, 64(17): 179201. doi: 10.7498/aps.64.179201
    [6] 靳艳飞, 李贝. 色关联的乘性和加性色噪声激励下分段非线性模型的随机共振. 物理学报, 2014, 63(21): 210501. doi: 10.7498/aps.63.210501
    [7] 张世功, 吴先梅, 张碧星. 基于迟滞应力应变关系的非线性声学检测理论与方法研究. 物理学报, 2014, 63(19): 194302. doi: 10.7498/aps.63.194302
    [8] 金左轮, 韩静, 张毅, 柏连发. 基于纹理显著性的微光图像目标检测. 物理学报, 2014, 63(6): 069501. doi: 10.7498/aps.63.069501
    [9] 刘洪臣, 李飞, 杨爽. 基于周期性扩频的单相H桥逆变器非线性现象的研究. 物理学报, 2013, 62(11): 110504. doi: 10.7498/aps.62.110504
    [10] 王萍, 潘跃. 基于显著性特征的大冰雹识别模型. 物理学报, 2013, 62(6): 069202. doi: 10.7498/aps.62.069202
    [11] 祝振敏, 曲兴华, 毕超, 贾果欣, 张福民. 基于LED阵列的彩色视觉检测光源色度特性研究. 物理学报, 2012, 61(2): 020702. doi: 10.7498/aps.61.020702
    [12] 宁方立, 董梁, 张文治, 王康. 谐振管内非线性驻波的有限体积数值算法. 物理学报, 2012, 61(19): 190203. doi: 10.7498/aps.61.190203
    [13] 侯祥林, 刘铁林, 翟中海. 非线性偏微分方程边值问题的优化算法研究与应用. 物理学报, 2011, 60(9): 090202. doi: 10.7498/aps.60.090202
    [14] 过榴晓, 徐振源. 关于非线性系统两类广义混沌同步存在性的研究. 物理学报, 2008, 57(10): 6086-6092. doi: 10.7498/aps.57.6086
    [15] 龚志强, 封国林. 基于非线性分析方法的多种代用资料的相似性研究. 物理学报, 2007, 56(6): 3619-3629. doi: 10.7498/aps.56.3619
    [16] 张政伟, 樊养余, 曾 黎. 一种精确检测未知弱复合周期信号频率的非线性融合方法. 物理学报, 2006, 55(10): 5115-5121. doi: 10.7498/aps.55.5115
    [17] 龚志强, 封国林, 董文杰, 李建平. 非线性时间序列的动力结构突变检测的研究. 物理学报, 2006, 55(6): 3180-3187. doi: 10.7498/aps.55.3180
    [18] 田晓东, 岳瑞宏. 推广的多分量费米型量子可导非线性Schr?dinger模型的可积性. 物理学报, 2005, 54(4): 1485-1489. doi: 10.7498/aps.54.1485
    [19] 徐云, 张建峡, 杜世培. 动力学系统中非线性项的跳跃随机性. 物理学报, 1991, 40(1): 33-38. doi: 10.7498/aps.40.33
    [20] 孙宗琦, 蒋方忻. 间隙原子非线性应力感生扩散的简化弹性偶极子模型. 物理学报, 1989, 38(10): 1679-1686. doi: 10.7498/aps.38.1679
计量
  • 文章访问数:  4381
  • PDF下载量:  287
  • 被引次数: 0
出版历程
  • 收稿日期:  2016-11-18
  • 修回日期:  2017-02-18
  • 刊出日期:  2017-05-05

/

返回文章
返回