Research Article
Inference Performance Comparison of Convolutional Neural Networks on Edge Devices
@INPROCEEDINGS{10.1007/978-3-030-76063-2_23, author={Sheikh Rufsan Reza and Yuzhong Yan and Xishuang Dong and Lijun Qian}, title={Inference Performance Comparison of Convolutional Neural Networks on Edge Devices}, proceedings={Science and Technologies for Smart Cities. 6th EAI International Conference, SmartCity360°, Virtual Event, December 2-4, 2020, Proceedings}, proceedings_a={SMARTCITY}, year={2021}, month={5}, keywords={Model compression Edge device Deep learning Internet of Things}, doi={10.1007/978-3-030-76063-2_23} }- Sheikh Rufsan Reza
Yuzhong Yan
Xishuang Dong
Lijun Qian
Year: 2021
Inference Performance Comparison of Convolutional Neural Networks on Edge Devices
SMARTCITY
Springer
DOI: 10.1007/978-3-030-76063-2_23
Abstract
With the proliferation of Internet of Things (IoT), large amount of data are generated at edge devices with an unprecedented speed. In order to protect the privacy and security of big edge data, as well as reduce the communications cost, it is desirable to process the data locally at the edge devices. In this study, the inference performance of several popular pre-trained convolutional neural networks on three edge computing devices are evaluated. Specifically, MobileNetV1 & V2 and InceptionV3 models have been tested on NVIDIA Jetson TX2, Jetson Nano, and Google Edge TPU for image classification. Furthermore, various compression techniques including pruning, quantization, binarized neural network, and tensor decomposition are applied to reduce the model complexity. The results will provide a guidance for practitioners when deploying deep learning models on resource constrained edge devices for near real-time and on-site learning.
