Predictive Latency Optimization for Generative AI: Jointly Optimizing Precomputation Time and User Waiting Latency

The increasing deployment of generative artificial intelligence (AI) systems has created substantial challenges in managing inference latency, particularly when large language models (LLMs) operate under fluctuating workloads, heterogeneous computational requirements, and strict user-facing service-level objectives. Predictive computing offers a potential mechanism for reducing future inference workload by anticipating user demand and preparing responses or reusable computational states before corresponding requests arrive. However, advance computation introduces an additional temporal optimization problem. Preparing a response requires computational time, and performing preparation too early, too late, or under inappropriate resource conditions may reduce its practical value. Moreover, predictive preparation can compete with active inference requests for GPU resources, memory bandwidth, and execution capacity, potentially increasing the waiting time of existing users. This paper introduces Predictive Latency Optimization for Generative AI (PLO-GAI), a conceptual framework that jointly optimizes advance computational preparation and user-facing inference latency. Unlike conventional inference optimization, which primarily reduces computation after requests arrive, and predictive computing, which primarily anticipates future computational demand, PLO-GAI explicitly models the temporal relationship between prediction initiation, preparation completion, future request arrival, residual online computation, and final response delivery. The framework integrates predictive request-arrival estimation, preparation-duration prediction, deadline-aware computational scheduling, latency-aware resource allocation, partial and complete response preparation, queue-aware execution control, and adaptive preparation cancellation. A unified multi-objective optimization formulation is developed to minimize expected user waiting latency and predictive preparation expenditure while satisfying GPU capacity, memory, response-quality, and service-level constraints. The framework distinguishes between computational time saved, computational time transferred to earlier periods, and user-visible waiting time actually avoided. It further investigates the trade-offs between preparation lead time, preparation completion probability, resource contention, prediction uncertainty, and request-specific computational requirements. An experimental methodology is proposed to compare predictive latency optimization with reactive inference, conventional caching, latency-unaware precomputation, and resource-aware predictive preparation. The proposed research does not assume that predictive preparation necessarily reduces end-to-end latency; its effectiveness depends on whether preparation can be completed before demand materializes and whether the remaining online computation is sufficiently reduced without degrading concurrent service. By treating precomputation duration and user waiting time as jointly controllable temporal variables, PLO-GAI establishes a distinct research direction for latency-aware predictive computing in generative AI infrastructure.