搜索

x

留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

基于马尔可夫决策模型的异构无线网络切换选择算法

梁潇 钱志鸿 田洪亮 王雪

引用本文:
Citation:

基于马尔可夫决策模型的异构无线网络切换选择算法

梁潇, 钱志鸿, 田洪亮, 王雪

Markov decision model based handoff selection algorithm for heterogeneous wireless networks

Liang Xiao, Qian Zhi-Hong, Tian Hong-Liang, Wang Xue
PDF
导出引用
  • 针对异构环境下不同业务类型用户对于接入网络的不同服务质量(quality of service,QoS)需求,该文提出了一种基于马尔可夫决策模型的切换选择算法.建立基于软件定义网络(software defined network,SDN)的异构无线网络架构,以实现对异构网络的通透控制.利用马尔可夫过程预测下一时刻的网络状态以得到采取动作后的一次回报,依据网络的不同状态属性针对实时用户和非实时用户分别构建立即回报函数,并采用层次分析法确定属性权重;基于状态动作对构建期望回报函数,采用逐次逼近的迭代方式得到使长期期望回报最大的切换策略.仿真结果表明,该方法针对不同业务类型用户均能选取最优切换策略,同时降低阻塞率,提高了用户的QoS和无线网络的资源利用率.
    Coexistence of multiple wireless access technologies will be an indicator of next-generation wireless network, and the integration of heterogeneous wireless networks will meet the needs of high-performance services for mobile users. According to unique quality of service (QoS) requirements of different service type users in heterogeneous environment, the Markov decision model based handoff selection algorithm is proposed in this paper. A heterogeneous wireless network architecture based on the software defined network (SDN) is established to realize the transparency control of heterogeneous networks. Network state information of heterogeneous wireless networks is mastered by SDN controller. It is responsible for scheduling network resources dynamically according to the performance characteristics of each network. If the network state information in equal interval is sampled, the next moment state of network is only related to the current network state and action, but it is not related to the historical state. The problem of handoff selection for heterogeneous wireless networks is modeled as a Markov process with discrete time and continuous state. To predict the next moment state of network by Markov process to obtain a reward, when the reward is positive, it represents the income; when it is negative, it represents the cost. An immediate reward function is constructed for real-time service and non real-time service users respectively according to their different state attributes of the network. Considering five state attributes of wireless network as follows:delay, delay jitter, bandwidth, error rate and network load, the immediate reward function is constructed with weighted summation. Due to the difference in attribute weight distribution among different service type users, the attribute weights are determined by the analytic hierarchy process. In the long term, the objective function which consists of immediate reward function sequence is used to measure future long-term rewards. Then expected reward function based on the state action pair is constructed to obtain the handoff strategy of the maximum expected return by the iterative method of successive approximation. The proposed Markov decision model based handoff selection algorithm is used in simulation of the Matlab platform. The simulation results show that the proposed method can select the optimal handoff strategy for different service type users and reduce the blocking rate, thereby improving the QoS of users and resource utilization of wireless networks.
      通信作者: 钱志鸿, dr.qzh@163.com
    • 基金项目: 国家自然科学基金(批准号:61371092)、国家自然科学基金青年科学基金(批准号:61401175)、吉林省重点科技攻关项目(批准号:20140204019GX)和长春市重大科技攻关计划(批准号:2014026/14KG021)资助的课题.
      Corresponding author: Qian Zhi-Hong, dr.qzh@163.com
    • Funds: Project supported by the National Natural Science Foundation of China (Grant No. 61371092), the Young Scientists Fund of the National Natural Science Foundation of China (Grant No. 61401175), the Key Science and Technology Program of Jilin Province, China (Grant No. 20140204019GX), and the Key Science and Technology Program of Changchun City, China (Grant No. 2014026/14KG021).
    [1]

    Falowo O E, Chan H A 2012 Eurasip J. Wirel. Comm. 221

    [2]

    Zhu K, Niyato D, Wang P 2010 Proceedings of IEEE Wireless Communications and Networking Conference Sydney, Australia, April 18-21, 2010 p1

    [3]

    Yan X, Šekerciğlu lu Y A, Narayanan S 2010 Comput. Networking 54 1848

    [4]

    Liu J, Xiong Q Y, Shi X, Wang K, Shi W R 2015 Chin. Phys. B 24 076401

    [5]

    Ahmed A, Boulahia L M, Gaiti D 2014 IEEE Commun. Surv. Tutorials 16 776

    [6]

    Hasib A, Fapojuwo A 2008 IEEE Trans. Veh. Technol. 57 2426

    [7]

    Kunarak S, Sulessathira R, Dutkiewicz E 2013 Proceedings of IEEE International Conference of Region 10 Xi'an, China, October 22-25, 2013 p1

    [8]

    Salem M, Ismail M, Misran N 2011 J. Appl. Sci. 11 336

    [9]

    Niyato D, Hossain E 2009 IEEE Trans. Veh. Technol. 58 2008

    [10]

    Naghavi P, Rastegar S H, Shah-Mansouri V, Kebriaei H 2016 IEEE Wirel. Commun. Lett. 5 52

    [11]

    Stevens-Navarro E, Martinez-Morales J D, Pineda-Rico U 2012 J. Appl. Res. Technol. 10 534

    [12]

    Wang N, Shi W X, Fan S S, Liu S X 2011 Proceedings of 2nd International Conference on Challenges in Environmental Science and Computer Engineering Haikou, China, December 14-15, 2011 p55

    [13]

    Liu K M 2014 J. Inf. Comput. Secor. 11 3373

    [14]

    Zhu S F, Liu F, Chai Z Y, Qi Y T, Wu J S 2012 Acta Phys. Sin. 61 096401 (in Chinese)[朱思峰, 刘芳, 柴争义, 戚玉涛, 吴建设2012物理学报61 096401]

    [15]

    Ning Z L, Song Q Y, Liu Y J, Wang F Z, Wu X Y 2014 Comput. Electr. Eng. 40 456

    [16]

    Ma B, Deng H, Xie X Z, Liao X F 2015 China Commun. 12 106

    [17]

    Ma B, Xie X Z, Liao X F 2015 J. Electron. Inform. Technol. 37 874 (in Chinese)[马彬, 谢显中, 廖晓峰2015电子与信息学报37 874]

    [18]

    Chen T, Matinmikko M, Chen X F, Zhou X, Ahokangas P 2015 IEEE Commun. Mag. 53 126

    [19]

    Wang H C, Chen S Z, Xu H, Ai M, Shi Y 2015 IEEE Network 29 16

    [20]

    Shen Y 2013 Chin. Phys. B 22 058902

    [21]

    Yang X L, Tan X Z, Guan K 2015 Acta Phys. Sin. 64 108403 (in Chinese)[杨小龙, 谭学治, 关凯2015物理学报64 108403]

    [22]

    Tsai C, Yang F N 2013 J. Hydraul. Eng. 139 1265

    [23]

    Fei R, Cui D W 2009 Acta Phys. Sin. 58 5133 (in Chinese)[费蓉, 崔杜武2009物理学报58 5133]

    [24]

    Alavipoor F S, Karimi S, Balist J, Khakian A H 2016 Global. J. Environ. Sci. Manage. 2 197

    [25]

    Marco W, Martijn V O 2012 Reinforcement Learning:State of the Art (Berlin:Springer) pp223-229

  • [1]

    Falowo O E, Chan H A 2012 Eurasip J. Wirel. Comm. 221

    [2]

    Zhu K, Niyato D, Wang P 2010 Proceedings of IEEE Wireless Communications and Networking Conference Sydney, Australia, April 18-21, 2010 p1

    [3]

    Yan X, Šekerciğlu lu Y A, Narayanan S 2010 Comput. Networking 54 1848

    [4]

    Liu J, Xiong Q Y, Shi X, Wang K, Shi W R 2015 Chin. Phys. B 24 076401

    [5]

    Ahmed A, Boulahia L M, Gaiti D 2014 IEEE Commun. Surv. Tutorials 16 776

    [6]

    Hasib A, Fapojuwo A 2008 IEEE Trans. Veh. Technol. 57 2426

    [7]

    Kunarak S, Sulessathira R, Dutkiewicz E 2013 Proceedings of IEEE International Conference of Region 10 Xi'an, China, October 22-25, 2013 p1

    [8]

    Salem M, Ismail M, Misran N 2011 J. Appl. Sci. 11 336

    [9]

    Niyato D, Hossain E 2009 IEEE Trans. Veh. Technol. 58 2008

    [10]

    Naghavi P, Rastegar S H, Shah-Mansouri V, Kebriaei H 2016 IEEE Wirel. Commun. Lett. 5 52

    [11]

    Stevens-Navarro E, Martinez-Morales J D, Pineda-Rico U 2012 J. Appl. Res. Technol. 10 534

    [12]

    Wang N, Shi W X, Fan S S, Liu S X 2011 Proceedings of 2nd International Conference on Challenges in Environmental Science and Computer Engineering Haikou, China, December 14-15, 2011 p55

    [13]

    Liu K M 2014 J. Inf. Comput. Secor. 11 3373

    [14]

    Zhu S F, Liu F, Chai Z Y, Qi Y T, Wu J S 2012 Acta Phys. Sin. 61 096401 (in Chinese)[朱思峰, 刘芳, 柴争义, 戚玉涛, 吴建设2012物理学报61 096401]

    [15]

    Ning Z L, Song Q Y, Liu Y J, Wang F Z, Wu X Y 2014 Comput. Electr. Eng. 40 456

    [16]

    Ma B, Deng H, Xie X Z, Liao X F 2015 China Commun. 12 106

    [17]

    Ma B, Xie X Z, Liao X F 2015 J. Electron. Inform. Technol. 37 874 (in Chinese)[马彬, 谢显中, 廖晓峰2015电子与信息学报37 874]

    [18]

    Chen T, Matinmikko M, Chen X F, Zhou X, Ahokangas P 2015 IEEE Commun. Mag. 53 126

    [19]

    Wang H C, Chen S Z, Xu H, Ai M, Shi Y 2015 IEEE Network 29 16

    [20]

    Shen Y 2013 Chin. Phys. B 22 058902

    [21]

    Yang X L, Tan X Z, Guan K 2015 Acta Phys. Sin. 64 108403 (in Chinese)[杨小龙, 谭学治, 关凯2015物理学报64 108403]

    [22]

    Tsai C, Yang F N 2013 J. Hydraul. Eng. 139 1265

    [23]

    Fei R, Cui D W 2009 Acta Phys. Sin. 58 5133 (in Chinese)[费蓉, 崔杜武2009物理学报58 5133]

    [24]

    Alavipoor F S, Karimi S, Balist J, Khakian A H 2016 Global. J. Environ. Sci. Manage. 2 197

    [25]

    Marco W, Martijn V O 2012 Reinforcement Learning:State of the Art (Berlin:Springer) pp223-229

  • [1] 孙瑛璐, 段延敏, 程梦瑶, 袁先漳, 张立, 张栋, 朱海永. 自拉曼混频黄绿波段三波长可切换激光. 物理学报, 2020, 69(12): 124201. doi: 10.7498/aps.69.20200324
    [2] 潘昕浓, 王革丽, 杨培才. 利用慢特征分析法提取层次结构系统中的外强迫. 物理学报, 2017, 66(8): 080501. doi: 10.7498/aps.66.080501
    [3] 罗小元, 李昊, 马巨海. 基于最小刚性图代数特性的无线网络拓扑优化算法. 物理学报, 2016, 65(24): 240201. doi: 10.7498/aps.65.240201
    [4] 贺志, 李莉, 姚春梅, 李艳. 利用量子相干性判定开放二能级系统中非马尔可夫性. 物理学报, 2015, 64(14): 140302. doi: 10.7498/aps.64.140302
    [5] 杨小龙, 谭学治, 关凯. 认知无线电网络中基于抢占式排队论的频谱切换模型. 物理学报, 2015, 64(10): 108403. doi: 10.7498/aps.64.108403
    [6] 滕启治, 谭欣, 武紫玉, 沈俊, 王海峰. 大型水轮发电机冷却方式综合评价方法的研究. 物理学报, 2015, 64(17): 178802. doi: 10.7498/aps.64.178802
    [7] 孙一杰, 张国良, 张胜修, 曾静. 一类异构多智能体系统固定和切换拓扑下的一致性分析. 物理学报, 2014, 63(22): 220201. doi: 10.7498/aps.63.220201
    [8] 尹文也, 何伟基, 顾国华, 陈钱. 模拟回火马尔可夫链蒙特卡罗全波形分析方法. 物理学报, 2014, 63(16): 164205. doi: 10.7498/aps.63.164205
    [9] 高超, 毕勤胜, 张正娣. 一个跃变电路切换系统的振荡行为及分岔机理分析. 物理学报, 2013, 62(2): 020504. doi: 10.7498/aps.62.020504
    [10] 余跃, 张春, 韩修静, 姜海波, 毕勤胜. 周期切换下Chen系统的振荡行为与非光滑分岔分析. 物理学报, 2013, 62(2): 020508. doi: 10.7498/aps.62.020508
    [11] 谢文贤, 许鹏飞, 蔡力, 李东平. 随机双指数记忆耗散系统的非马尔可夫扩散. 物理学报, 2013, 62(8): 080503. doi: 10.7498/aps.62.080503
    [12] 林方, 胡丹青, 李乐乐. 用一种分数阶算法研究非马尔可夫过程中阻尼与涨落的竞争机制. 物理学报, 2013, 62(12): 120503. doi: 10.7498/aps.62.120503
    [13] 蔡诚俊, 方卯发, 肖兴, 黄江. 非马尔可夫环境下经典场驱动Jaynes-Cummings模型中原子的熵压缩. 物理学报, 2012, 61(21): 210303. doi: 10.7498/aps.61.210303
    [14] 柴争义, 刘芳, 朱思峰. 混沌量子克隆优化求解认知无线网络决策引擎. 物理学报, 2012, 61(2): 028801. doi: 10.7498/aps.61.028801
    [15] 朱思峰, 刘芳, 柴争义, 戚玉涛, 吴建设. 简谐振子免疫优化算法求解异构无线网络垂直切换判决问题. 物理学报, 2012, 61(9): 096401. doi: 10.7498/aps.61.096401
    [16] 柴争义, 刘芳, 朱思峰. 混沌量子克隆算法求解认知无线网络频谱分配问题. 物理学报, 2011, 60(6): 068803. doi: 10.7498/aps.60.068803
    [17] 郑力明, 刘颂豪, 王发强. 非马尔可夫环境下原子的几何相位演化. 物理学报, 2009, 58(4): 2430-2434. doi: 10.7498/aps.58.2430
    [18] 刘扬正, 姜长生. 关联可切换超混沌系统的构建与特性分析. 物理学报, 2009, 58(2): 771-778. doi: 10.7498/aps.58.771
    [19] 费蓉, 崔杜武. 马尔可夫随机过程中移动对象的空间特征分析及近似逼近研究. 物理学报, 2009, 58(8): 5133-5141. doi: 10.7498/aps.58.5133
    [20] 王焕元, 张寿恭, 潘孝硕. 用磁分析法观察某些铁-镍-铝合金的脱溶过程. 物理学报, 1960, 16(4): 214-228. doi: 10.7498/aps.16.214
计量
  • 文章访问数:  3088
  • PDF下载量:  234
  • 被引次数: 0
出版历程
  • 收稿日期:  2016-07-26
  • 修回日期:  2016-09-07
  • 刊出日期:  2016-12-05

基于马尔可夫决策模型的异构无线网络切换选择算法

  • 1. 吉林大学通信工程学院, 长春 130012;
  • 2. 东北电力大学信息工程学院, 吉林 132012
  • 通信作者: 钱志鸿, dr.qzh@163.com
    基金项目: 国家自然科学基金(批准号:61371092)、国家自然科学基金青年科学基金(批准号:61401175)、吉林省重点科技攻关项目(批准号:20140204019GX)和长春市重大科技攻关计划(批准号:2014026/14KG021)资助的课题.

摘要: 针对异构环境下不同业务类型用户对于接入网络的不同服务质量(quality of service,QoS)需求,该文提出了一种基于马尔可夫决策模型的切换选择算法.建立基于软件定义网络(software defined network,SDN)的异构无线网络架构,以实现对异构网络的通透控制.利用马尔可夫过程预测下一时刻的网络状态以得到采取动作后的一次回报,依据网络的不同状态属性针对实时用户和非实时用户分别构建立即回报函数,并采用层次分析法确定属性权重;基于状态动作对构建期望回报函数,采用逐次逼近的迭代方式得到使长期期望回报最大的切换策略.仿真结果表明,该方法针对不同业务类型用户均能选取最优切换策略,同时降低阻塞率,提高了用户的QoS和无线网络的资源利用率.

English Abstract

参考文献 (25)

目录

    /

    返回文章
    返回