An investigation of challenges in the machine learning lifecycle and the importance of MLOps: A survey

Quite recently, considerable attention has been paid to developingartificial intelligence and data science areas. This has been drivenby scientific advances and the growing number of software andservices that are popularizing machine learning techniques andalgorithms and driving people with less knowledge in areas suchas statistics and mathematics to create their predictive models. Asa result, the machine learning field is no longer only scientificand has aroused the interest of companies from different domains.These events led to the emergence of multiple tools such as Scikit-Learn, Tensorflow, Keras, Pycaret, and a vast number of cloud-basedmachine learning services that provide an acceleration in the developmentof predictive models at speeds never seen. However, manychallenges remain in operationalizing and maintaining machinelearning-centered products, making many business initiatives frustrated.In this scenario, practical experience shows that machinelearning is only a slice of a more extensive set of practices andtechnologies necessary to build solutions in this area. In this paper,the main goal is to identify the challenges currently faced by datascientists in developing Machine Learning-centric products andhow Machine Learning Operations can support overcoming them.For this purpose, a survey was conducted that collected answersfrom 66 Brazilian professionals in data science. From the challengesidentified, the importance of Machine Learning Operations practicesas an integrated part of the Machine Learning lifecycle wasexplored. Finally, this work contributes to filling the gap in MachineLearning Operations in daily activities involving data science andadvancing this research field in Brazil.