From Text to Taste: Advancing Smart Appliances with Multilingual Recipe Interpretation

In this work, we leverage pre-trained language models (PLM) to predict cooking parameters directly from text-based recipes. We explore a multilingual multi-task setting, where we train a single transformer-based model to handle six Indo-European languages and three tasks that identify cooking parameters: Nutrient composition categorization, Cooked product aspect, and Moisture reduction levels. Based on empirical analysis, our proposed method improves over a strong baseline by an average of 0.76% on the three tasks. We show that this multilingual multitask setting is particularly beneficial for low-resource languages, increasing the performance by as much as 7.4% for the harder tasks. We then show that the resulting model is capable of zero-shot cross-lingual generalization: the model can effectively be applied to languages for which it has not seen any labeled examples. We find that leveraging unlabeled cooking recipes to continue the pre-training phase of the model further increases its zero-shot performance by exposing it to the food domain. We report zero-shot enhancements per task equal to 0.82%, 8.45%, and 8.95% respectively. Our work shows that transformer-based models are capable of effectively interpreting and predicting diverse cooking-related parameters from text, which can have implications in the development of smart kitchen technologies such as smart appliances.