CRSP: Emulating Human Cooperative Reasoning for Intelligible Story Point Estimation

Software effort estimation plays a critical role in software project development. Inaccurate cost estimation can impact progress and result in budget overruns. The story point estimation technique is a commonly used practice in agile software development for the estimation of software development effort. It allows for the evaluation of relative task workloads by analyzing task titles and descriptions. In previous studies, researchers have mainly focused on providing story point estimation results by task titles. However, in practical scenarios, users are often unable to provide task titles as precise as those found in the training dataset, leading to inaccurate estimation results. To address this problem, we propose a Cooperative Reasoning Story Point estimation method (CRSP). We approach the estimation problem as a question-and-answer challenge, addressing it through a framework of model construction, Monte Carlo Tree search, and model inference. In the model construction phase, we train a generator responsible for generating problem-solving reasoning paths and employ verifier to score the quality of these reasoning paths. During the Monte Carlo Tree search stage, we execute MCTS using generator and verifier to generate candidate solutions. In the final model inference phase, we employ a solver to derive the ultimate answer. To evaluate the effectiveness of CRSP, we modify and adapt the well-known JIRA dataset to make it more compatible with the input format of the question-answering model. The new JIRA dataset contains 21,082 issues from 16 open-source software projects. Across 16 open-source projects, the mean absolute error of CRSP is lower than other baseline methods. In contrast to the traditional regression and classification methods, we pioneer the use of question-and-answer method to address the issue, opening up new directions for future research.