Operating Condition Recognition of Industrial Flotation Processes Using Visual and Acoustic Bimodal Autoencoder With Manifold Learning

The real-time recognition of operating conditions is always critical to ensuring the efficient and stable operation of industrial flotation processes. Although the widespread use of smart devices enables the availability of multimodal data in flotation processes, recognizing operating conditions using cross-modal data information is still challenging due to the modality gap. To address this issue, this article first proposes an innovative bimodal manifold autoencoder model to predict interested quality variables from the perspective of visual modality and auditory modality. Specifically, the well-designed intra- and intermodal manifold regularization constraints are introduced to fully learn the intrinsic manifold features within each modality and the interdependencies across modalities, thereby enhancing the cross-modal data representation ability of the developed prediction models. Then, based on the prediction values of quality variables, an adaptable multimodal fuzzy decision inference module is designed to recognize the operating conditions while counteracting the influence of fluctuations in feedstock properties. Finally, extensive experiments are conducted on two industrial flotation process datasets at distinct periods to validate the superiority of the proposed methods in terms of quality prediction and operation condition recognition tasks.