Answers are grounded in this paper and cite their sources.
Learning strategies from raw image inputs is an important problem in reinforcement learning. However, unlike the deep convolutional networks widely used in the computer vision field, the convolutional neural networks (CNNs) used in reinforcement learning often have smaller scales. Therefore, in this paper, we explore the use of deep visual models pre-trained on external datasets as feature extractors, and apply them without fine-tuning for downstream tasks. Finally, we evaluate our method on robot control tasks. Even when using unprocessed raw images as input and without observing any frame of the simulation environment during training, our pre-trained feature extractor can still match the ground truth feature, and the model converges faster in downstream tasks. We also investigate the performance of using outputs from different depth hidden layers as pre-trained features in downstream tasks, and analyze the hierarchical relationship of features and their performance in control tasks.
No takes yet. Share an insight, caveat, or question.
Wang et al. (2024) studied this question.