Answers are grounded in this paper and cite their sources.
Demonstrates high accuracy in classifying content properties from raw byte input, implying practical use in various applications.
We introduce pico-type, a tiny byte-level multi-head content classifier with approximately 1.5 million parameters that simultaneously predicts seven content properties from raw UTF-8 bytes in a single forward pass. Operating directly at the byte level with no tokenizer or pretrained embeddings, pico-type classifies coarse type, modality, subtype, code language, text language, file MIME type, and risk flags (API keys, JWTs, passwords). The model uses a lightweight trunk of byte embed- ding, convolutional blocks, bidirectional attention with rotary position embeddings, and a statistical pooling layer feeding seven Matryoshka-style classification heads. With four tiered variants ranging from 16 to 576 dimensions and ONNX export under 210 KB, pico-type achieves 100% accuracy on coarse, modality, file MIME, and risk detection, 98.4% on subtype, 100% on text language, and 53.9% on code language (62-way) through a mixed training approach combining synthetic data with real GitHub code samples. On a hand-curated real-world test set spanning 21 diverse inputs, the model achieves 52.4% overall accuracy across all heads, demonstrating practical viability beyond synthetic benchmarks. The model is designed for on-device deployment in CLIs, browser extensions, and MCP servers.
No takes yet. Share an insight, caveat, or question.
Kishore et al. (2026) studied this question.
Synapse has enriched 8 closely related papers on similar clinical questions. Consider them for comparative context: