Evaluating Prompt Engineering for Large Language Models-Based Translation

Large language models (LLMs) have introduced new possibilities for machine translation (MT) by allowing translation outputs to be shaped through natural-language instructions. However, evidence about how different prompt configurations affect translation quality relative to established neural machine translation (NMT) systems remains limited, particularly for Indonesian–English translation. This exploratory mixed-methods comparative study examines three prompt configurations applied to ChatGPT and compares the resulting outputs with DeepL and Google Translate across six tourism, promotional, and proverbial texts. Translation outputs were assessed computationally using cross-lingual cosine similarity from text-embedding-3-large as an exploratory indicator of semantic proximity and qualitatively through evaluations by three professional translators focusing on accuracy, appropriateness, naturalness, and discourse style. In the observed sample, DeepL and the baseline-prompt condition tended to obtain higher similarity scores for factual tourism texts, whereas contextual- and persona-prompt outputs received comparatively favorable expert evaluations in selected promotional and culturally embedded cases. Contextual prompting also produced additions beyond the wording shown in the source excerpts, indicating a trade-off between rhetorical adaptation and source fidelity. Because the corpus is small, the prompt conditions bundle several instructional features, repeated generations were not documented, and some reproducibility metadata were not archived, the findings are interpreted descriptively rather than as causal or statistically generalizable effects. The study therefore highlights a genre- and criterion-sensitive relationship among prompt configuration, semantic preservation, and functional translation quality, while underscoring the need for human verification of adaptive LLM translations.