In-House Integrated Big Data Management Platform for Exploration and Production Operations Digitalization: From Data Gathering to Generative AI through Machine Learning Implementation Using Cost-Effective Open-Source Technologies - Experienced Mature Workflow

Abstract The objective of this study is to perform an in-depth assessment of the In-house Data Integration Platform (IDIP). This solution, crafted by local engineers, is tailored to fit the purpose of enhancing digitalization endeavors within the company. Notably, the platform's distinctiveness lies in its role as both a culmination of local engineering ingenuity and a valuable input source for various commercial software applications. The IDIP employs a rigorous multi-disciplinary approach, integrating in-house programming, customized developed algorithms, and statistical methods alongside machine learning technologies. The evaluation methodology focuses on systematically centralizing, collecting, and structuring data from diverse sources. This includes a spectrum from Excel files and SQL databases to extensive big data, such as videos, hard files etc., spanning various departments. This is coupled with the identification of shortcomings in existing commercial tools to cover. The ultimate objective is to efficiently address the daily operational requirements of the company's technical team, minimizing efforts and time. The introduction of the locally developed platform has led to significant results, promoting increased teamwork and integration among various teams. The platform's impact is evident in its efficient handling and reporting of oil and gas reserves for over 400 fields with more than 500 different reservoirs. This achievement has substantially reduced completion time from over a month to about an hour. Additionally, the IDIP offers extra advantages, such as accommodating up to 500 simultaneous users engaging in different tasks with distinct input systems. This capability caters to a diverse user base and ensures smooth operations. In terms of production oversight, the platform serves both technical and non-technical users, providing a quick identification of any discrepancies in production allocation. The structured data provided by the platform is beneficial for data science and large language models, enabling easy utilization and contributing to efficient and organized data handling. The platform also establishes a real-time feedback loop, allowing swift reporting of suggestions or improvements. The IDIP not only improves multidisciplinary workflows but also plays a crucial role in preserving and capturing local engineering ingenuity. This study highlights its pivotal role in shaping a shared vision of subsurface scenarios. A distinctive feature of the platform lies in its seamless integration of machine-learning algorithms and data-driven models, providing a novel, flexible, and locally crafted contribution to the digitalization landscape. Additionally, the platform leverages the ingenuity of local engineers by depending on open-source libraries and frameworks, allowing for easy customization without the need for cumbersome workarounds.