Ghost-Score-Driven Multi-Pipeline Face Swap with 3D-Aware Visible-Surface Compositing and Apple-Silicon-Native Runtime

We present a hybrid, on-device face-swap frame-work that combines classical 2D warping, a 3D Morphable Model (3DMM) fit, neural face restoration, and a purpose-built ghost-face detector for runtime-adaptive routing. Unlike single-shot GAN swap networks, our system runs multiple candidate pipelines per request—a classical compositing transplant, an inswapper synthesis path, and a 3D-aware HardPoseReplace-Mode—and selects the winner by an ArcFace-derived ghost score g = cos(fout,ftgt)−cos(fout,fsrc), where f· are 512-D embed-dings. The ghost score directly measures whether the underlying target identity still leaks through the swap. When g exceeds a routing threshold, the system automatically invokes target-identity suppression (Telea and Navier–Stokes inpainting and bi-lateral low-frequency erasure), z-buffered visible-surface masking via 3DDFA V2, Depth-Anything-V2-derived foreground-occluder masks, and SAM 2 promptable mask refinement. Final boundary blending adaptively selects among Poisson NORMAL CLONE, MIXED CLONE, and a 5-band Laplacian-pyramid fusion based on the gradient-energy ratio across the mask. The pipeline is engi-neered for Apple Silicon: GFPGAN runs on Metal Performance Shaders (MPS), ONNX models run natively on arm64, and a request-serialising asyncio lock plus memory-pressure watchdog (24 GB RSS / 20-swap threshold) keep a single Python process stable across long sessions. On a cross-identity HD portrait test pair of two Unsplash-licensed real-photo portraits used under the Unsplash License (Subject A onto a different Subject B at 3600×2400), our Pro mode delivers an ArcFace identity-to-source cosine of 0.896, an identity-to-target cosine of 0.048, and a ghost score of−0.848 on an Apple M4 Max, while preserving the target subject’s hair, clothing and background. Across a 200-pair cross-identity benchmark spanning pose differences 1.3–140.6◦, the raw inswapper baseline reaches a 89.9% match rate, our AI Synthesis reaches 89.0%, and Classical Compositing reaches 75.9% with the lowest LPIPS (AlexNet and VGG) perceptual distance to the source. Against an external SimSwap- 256 baseline on the same 200 pairs, our pipelines lead by ∆αs ≥+0.36 in paired comparison. On the full 200-pair set, our Pro mode reaches αs = 0.784 with a 92.5% match rate, paired ∆αs(Pro−Classical) = +0.150±0.031 (95% CI, Wilcoxon p=1.2×10−21). On the 10 hardest pairs (those where Classical’s ghost score gCl ≥−0.05) Pro mode rescues 10/10 with mean ∆αs = +0.732±0.072 (95% CI), lifting αs from +0.08 to 0.811. This is the canonical demonstration of the multi-pipeline routing- by-ghost-score architecture. Index Terms—face swap, ghost detection, 3D Morphable Model, Apple Silicon, Poisson blending, Laplacian pyramid, ArcFace, MediaPipe FaceMesh, BiSeNet, Depth Anything, SAM 2. Preprint. Independent research; not yet peer-reviewed. Code available at https://github.com/Kayariyan28/Ghost-Score-Face-Swap