Abstract We introduce DV4 (Dual-Vocab 4-Bit), a novel large language model weight encoding scheme in which each weight is represented using 4 bits: 3 bits encoding a ternary value in -1, 0, +1, and a 4th flip bit that inverts the polarity of non-zero weights when set. Topic-specific binary masks over the flip bits allow a single set of ternary weights to exhibit qualitatively different behaviour across multiple knowledge domains without modifying the weights themselves. We term this mechanism topic-conditional weight reinterpretation. We train a 558M parameter DV4 transformer on four topics (mathematical reasoning, general language, programming, and science) using a topic-conditional training schedule, and evaluate cross-topic contamination via a perplexity-based bleed test across all 16 data/mask combinations. We observe topic specificity scores of +0. 637 (math), +39. 47 (general), +46, 005, 858 (code), and +916. 30 (science), with a mean of +11, 501, 703. The code topic produces a perplexity of 84. 7 billion under the math mask versus 614 under its correct mask — a specificity ratio exceeding 46 million. These results provide strong empirical evidence that topic-conditional weight reinterpretation via frozen flip bit masks is a viable and highly effective mechanism for multi-domain LLM specialisation at zero additional parameter cost at inference time.
Peter Norman (Sun,) studied this question.