Integration of artificial intelligence technologies in data engineering: Challenges and prospects in the modern information environment

The integration of artificial intelligence technologies into data engineering gained significant relevancy in the context of constantly growing volumes and complexity of data, which requires innovative approaches to processing and analysis. The goal of the present study is to conduct a deep analysis of the implementation of artificial intelligence into data engineering with a focus on the challenges occurring and perspectives of this process. Research methods, such as analysis methods, comparison, systematisation, and systemic approach, were used for an objective study of this phenomenon and revealing key aspects of this topic. Analysis revealed key challenges, that include variety and instability of data, the importance of standardisation as well as ensuring security of big data amounts. The importance of ethical aspects is underlined, and perspectives on automation of analytical processes and improving prognostic analysis were also determined. According to the results, employment of common standards improves the consistency of approaches, whereas improved algorithms accelerate the processing of big data amounts. Employment of such technology as Apache Hadoop and Spark for processing big data amounts and step-by-step introduction of artificial intelligence is also useful. Increased decision explication also improves their understanding, simplifying interaction between experts and interested parties, and simultaneously creating conditions for effective implementation and employment of integrated artificial intelligence systems in data engineering. The compilation of ethical standards and legal mechanisms creates an opportunity for responsible and balanced employment of these technologies, ensuring trust and ethical compliance in the process of their implementation into various spheres of human activity. These results determine perspectives for the development of this sphere and highlight its importance in a modern informationbased society. Integration of artificial intelligence into data engineering expands capabilities of automating analytical processes, ensuring accurate predictions, and reducing manual labour expenses, creating opportunities for effective management and reasoned decision-making in the data processing sphere