Zero-shot and few-shot multimodal plastic waste classification with vision-language models

The construction sector is a large consumer of plastic, generating substantial volumes of plastic waste. Effective recycling of this waste requires accurate classification, as different plastic materials undergo distinct recycling processes to retain their value. Existing approaches for plastic waste classification predominantly utilise fully supervised deep learning models, which require extensive labelled datasets. Such datasets are challenging to compile and label, and restrict the model's generalisability and scalability, as identifying new plastic classes necessitates additional labelled data. Hence, this paper explores the application of Vision-Language Models (VLMs) for classifying construction and demolition plastic waste by resin type. VLMs offer the advantage of generalising to unseen categories through zero-shot learning, utilising only the natural language descriptions of visual concepts. Comprehensive experiments are conducted to evaluate the efficacy of advanced VLMs, in zero-shot classification of plastic waste, using natural language descriptions of plastic categories. Furthermore, image and textual modalities are integrated in multimodal few-shot learning frameworks and the outcomes are compared with those from data-intensive, fully supervised baselines to assess a balance between accuracy, scalability, and data efficiency. The findings demonstrate that VLMs effectively classify end-of-life plastics with minimal to no training data, with the best-performing models achieving 70.15 % accuracy in zero-shot classification and improving to 85.07 % with multimodal few-shot learning, showcasing substantial improvements in data efficiency and scalability.