Deep Reinforcement Learning-Based Edge Caching in Heterogeneous Networks

Abstract

With the increasing number of mobile device users worldwide, utilizing mobile edge computing (MEC) devicesclose to users for content caching can reduce transmission latency than receiving content from a server or cloud. However, because MEC has limited storage capacity, it is necessary to determine the content types and sizesto be cached. In this study, we investigate a caching strategy that increases the hit ratio from small base stations(SBSs) for mobile users in a heterogeneous network consisting of one macro base station (MBS) and multipleSBSs. If there are several SBSs that users can access, the hit ratio can be improved by reducing duplicate contentand increasing the diversity of content in SBSs. We propose a Deep Q-Network (DQN)-based caching strategythat considers time-varying content popularity and content redundancy in multiple SBSs. Content is stored inthe SBS in a divided form using maximum distance separable (MDS) codes to enhance the diversity of thecontent. Experiments in various environments show that the proposed caching strategy outperforms the othermethods in terms of hit ratio.