Attribute-based people search: open attribute recognition for person re-identification in surveillance and security

The extraction and use of textual descriptions to identify and retrieve individuals in a given video footage is known as person attribute recognition for person re-identification (ReID). This approach bridges the gap between natural language processing (NLP) and computer vision, expanding the research field with diverse applications, including surveillance, security, and autonomous driving technologies. Current person re-identification (ReID) systems often rely on an image as input, which limits their usefulness in cases where only a textual description is available, such as in law enforcement investigations or searches for missing persons. As a result, effective retrieval is obstructed when visual data is scarce or unavailable. Moreover, current text-based ReID methods, which depend on full sentences to find the target person, can be computationally expensive, especially when handling long surveillance videos. Attribute-based ReID offers a faster alternative; however, its dependence on a fixed set of predefined attributes restricts the system’s ability to handle variations found in real-world descriptions, thus reducing robustness. This work addresses these limitations by proposing a novel framework that uses open attributes in person re-identification and mitigates the image dependency of traditional methods. Crucially, our work’s novelty lies in introducing a unique text-to-text similarity approach for person re-identification, operating entirely within a semantic textual open attribute space. This framework consists of three main modules. A natural language processing (NLP) module is implemented to process the input textual query from the user and extract the keywords that describe the target person. A person detection module localizes individuals within the video frames, generating the search gallery. A person open attribute recognition (POAR) module is used to generate open attributes from each given gallery image, compare the text query attributes with the gallery image attributes using cosine similarity, and retrieve the best-ranked image for the user. Experiments demonstrate the effectiveness of our proposed framework, achieving a rank-1 accuracy of 84.8% in identifying and retrieving individuals from video data. These results outperform state-of-the-art methods, thus validating the potential of the approach in real-world applications. For training and evaluation, we employed the PA-100k dataset, accessible at https://www.kaggle.com/datasets/yuulind/pa-100k .