RunTyper: Enhancing Deep Type Inference Using Dynamic Analysis for Python

Dynamic languages like Python are highly valued for the flexibility they offer, primarily due to their dynamic typing system. However, this flexibility often comes at a cost: type-related errors typically manifest only at runtime, which can result in elusive bugs. This scenario highlights the urgent need for effective automatic type inference strategies. Recently, there has been significant progress in the development of deep learning-based methods targeting the issues. Despite their potential, these methods face considerable limitations in the context of rare types (predominantly user-defined types in Python), which are often underrepresented in training datasets. To tackle the inherent unpredictability of dynamic characteristics in Python, we present an innovative framework named RunTyper. RunTyper uniquely integrates dynamic analysis with a sophisticated framework that combines dynamic instrumentation and static rules. This hybrid approach is utilized to validate and refine predictions made by deep neural networks. When compared with Type4Py, a deep learning-based type inference tool, the experiment results indicate that RunTyper achieves a significantly higher accuracy rate of 53.7%, whereas Type4Py attains only 41.9%. Additionally, a series of experiments demonstrate that RunTyper significantly enhances performance in user-defined types. When compared to HiTyper, which similarly seeks to refine deep type inference but is limited to static rules, RunTyper exhibits a notable improvement, increasing accuracy from 17.0% to 22.9%, an enhancement rate of 35%. Moreover, the experiments indicate that RunTyper improves the accuracy of GPT-4 in processing user-defined types by a remarkable 17%. These findings emphasize RunTyper’s proficiency in refining and enhancing the results produced by deep prediction models, as well as its ability to accurately infer rare types that are traditionally difficult to predict.