A reinforcement learning-driven framework for real-time human-computer interaction: latency-aware composite reward optimization based on PPO

While Large Language Models (LLMs) exhibit remarkable zero-shot learning capabilities, their deployment in real-time interactive systems is constrained by sparse terminal feedback in long-sequence generation, unstable policy updates, and the computational overhead associated with verbose outputs. To address these challenges, this paper formalizes autoregressive text generation as a discrete-time Markov decision process (MDP) and proposes a latency-aware composite reward optimization framework based on proximal policy optimization (PPO). The core contribution of the proposed method is a multidimensional sequence-level terminal reward that jointly evaluates semantic coherence, lexical diversity, and computational cost. Specifically, a Natural Language Inference (NLI) cross-encoder evaluates the semantic consistency between the input context and the complete generated response, while Shannon entropy and Distinct-N metrics encourage lexical diversity. In addition, a hardware-aware penalty based on generated sequence length and measured inference latency discourages unnecessarily verbose outputs and promotes computationally efficient generation. During policy optimization, the terminal composite reward is propagated through generalized advantage estimation, while a token-wise Kullback–Leibler divergence term regularizes the updated policy against the reference policy. This formulation does not introduce token-level semantic rewards; instead, it combines informative sequence-level evaluation with step-wise policy regularization to improve empirical training stability. We evaluated the proposed framework on a large-scale hybrid corpus containing approximately 320,000 dialogue turns, including real-world English interaction data. Compared with the standard PPO baseline, the proposed method exhibited faster empirical convergence and lower variability in the observed training trajectories. It achieved a BLEU-4 score of 22.87 and a Distinct-2 score of 21.59%, while reducing the average inference latency to 42.8 milliseconds per token under the reported hardware configuration. These results demonstrate the potential of latency-aware composite reward optimization to balance response quality, lexical diversity, and inference efficiency in real-time domain-specific LLM applications.