Promoting knowledge recommendation in innovative engineering design: a BERT-GAT-based patent representation learning approach

Since innovation in complex product design hinges on thorough engineering knowledge application, high-quality patent recommendations foster innovation in engineering design. However, many patent knowledge recommendation studies perform patent analysis without comprehensive exploration and proper organisation of knowledge, causing a superficial understanding of patents and returning arbitrary results. To mitigate this issue, a deep learning-based approach for patent representation learning and knowledge recommendation is proposed. First, a four-dimensional patent knowledge model is defined to formalise the patent attributes that critically affect the engineering design outcomes, namely patents' domain(D), function(F), technology(T) and citation(C). Second, to exploit patent knowledge from their content and citation relationships, a representation learning approach integrating Bidirectional Encoder Representations from Transformers(BERT) and Graph Attention Network(GAT) is introduced. Thereafter a patent knowledge space is established in which each patent is characterised by the function, technology, and citation embeddings. Third, a knowledge requirement space is also constructed by vectorising a designer's search query via BERT model and linking it to a requirement-representing patent based on similarity. Finally, a recommender prototype is developed and showcased by the knowledge recommendation in sealing structure design tasks. Comparative experiments and application cases validate the effectiveness of our method in patent representation learning and knowledge recommendation.