Evaluating foundation model integration strategies for detecting PII in java software engineering pipelines

Context: Java source code may inadvertently embed personally identifiable information (PII), creating privacy, security, and regulatory compliance risks in contemporary software engineering pipelines. Objective: We empirically compare three architectural strategies for entity-level PII detection in Java source code: a classifier-only pipeline based on transformer models, a hybrid classifier plus large language model (LLM) pipeline in which an open-weight LLM judges classifier-generated candidates, and an LLM-centered structured-extraction pipeline with deterministic validation and sanitization. Method: We evaluate the three pipelines on a synthetic Java dataset using shared preprocessing, value-level matching, and micro-averaged precision, recall, and F ₁ -score. The experiments use controlled configurations, fixed prompt templates, deterministic post-processing, and archived artifacts to support reproducibility and auditability. Results: Individual classifier baselines showed limited recall in isolation, while union-based ensemble aggregation increased coverage at the cost of additional false positives. The hybrid architecture reduced some false positives, but also systematically removed baseline true positives, producing precision-oriented filtering behavior that prioritizes selectivity over coverage. In contrast, the LLM-centered structured-extraction pipeline achieved the most favorable precision–recall balance among the evaluated strategies when structured-output compliance was stable and deterministic sanitization was enforced. Conclusions: The results indicate that architectural integration strategies and pipeline constraints substantially shape PII detection behavior, often beyond the isolated choice of individual models. Under controlled conditions, open-weight LLMs can support privacy-oriented source code analysis when embedded in reproducible, auditable, and validation-aware pipelines that explicitly monitor recall loss and structured-output robustness.