A Stacked Multi-Layered Perceptron - LLM Model for Extracting the Relations in Textual Descriptions

The automated extraction of the relations among the concepts in textual descriptions is important for problems that require creating implementations of those descriptions, e.g., synthesizing engineering designs and computer code. However, relation extraction remains challenging despite significant recent progress in Natural Language Processing. This paper proposes a novel, two-layered method to create structural representations of the relations in a text. Inspired by work in neuroscience, an upper layer implemented as a multi-layer perceptron models the behavior of closed class words (words with a fixed meaning), like prepositions and conjunctions. The lower layer prompts a Large Language Model to extract the nouns and verbs, which are then introduced into the structural representation produced by the upper layer. Experiments show an average improvement of 13.6% as compared to using only LLMs.