Data job requirement characteristics analysis based on large language model and K-means clustering

This paper aims to analyze the characteristics of data-related job requirements using a Large Language Model (LLM), providing decision support for the construction of data-related courses in universities. Using Zhaopin.com as the data source and employing a key attribute extraction method based on ChatGLM3 along with prompting engineering techniques, this study extracts and clusters key attributes from job descriptions. It analyzes the characteristics of data-related job requirements in terms of competencies, fields of expertise, and tools. The research found that the quality of key attribute extraction using ChatGLM3 improved by 54.1% compared to dictionary-based methods, significantly reducing the workload of manual annotation. Additionally, the analysis revealed commonalities and differences in the demand features of data analysis, data engineering, and data operations roles across six dimensions: soft skills, tools, specialties required, educational background, experience, and salary.