摘要
基于卷积神经网络(CNN)的移动应用程序通常处理的是计算密集型任务,然而传统的云推理和端推理方式在低时延和高准确率方面仍面临着挑战。针对以上问题,提出了一种基于边端协同的CNN推理框架,它能让一个端设备和多个边缘服务器协同工作以提供CNN推理服务。该方法综合考虑高度动态的网络带宽和设备负载情况,分步决策出模型多个最佳分割位置以优化计算和通信权衡。基于硬件测试平台的实验评估表明,相较于3种流行的CNN推理方法,该方法实现了14.3%~67.5%的推理加速,提升边缘服务器的计算资源利用率100%。
Mobile applications based on convolutional neural networks(CNN)usually deal with computing intensive tasks.However,traditional cloud inference and end inference still face challenges of low latency and high accuracy.In this regard,a CNN inference framework based on the edge⁃end collaboration is proposed.It enables one end device and multiple edge servers to work together to provide CNN inference services.This method considers the highly dynamic network bandwidth and equipment load conditions,and determines multiple optimal segmentation points of the model stepwise to optimize the communication and computing trade⁃offs.The results of the evaluation on the hardware test platform show that method achieves 14.3%—67.5%inference acceleration and improves the utilization rate of the edge server’s computing resources by 100%,compared to the results of the three popular CNN inference methods.
作者
郭永安
周金粮
王宇翱
GUO Yongan;ZHOU Jinliang;WANG Yuao(School of Communications and Information Engineering,Nanjing University of Posts and Telecommunications,Nanjing 210003,China;Engineering Research Center of Health Service System Based on Ubiquitous Wireless Networks,Ministry of Education,Nanjing 210003,China;Edge Intelligence Research Institute,Nanjing University of Posts and Telecommunications,Nanjing 210003,China)
出处
《南京邮电大学学报（自然科学版）》
北大核心
2023年第3期68-77,共10页
Journal of Nanjing University of Posts and Telecommunications：Natural Science Edition
基金
江苏省前沿引领技术基础研究专项(BK20202001)
江苏省重点研发计划(社会发展)(BE2020721)资助项目。
关键词
协同CNN推理
分步决策
低时延
计算资源利用率
collaborative CNN inference
step⁃by⁃step decision⁃making
low latency
computing resource utilization
作者简介
郭永安,男,博士,教授,guo@njupt.edu.cn。
