Answers are grounded in this paper and cite their sources.
A solitary Visual Perception Module is required to facilitate image-level recognition, object localisation, long-tail instance segmentation, scene parsing and promptable mask transfer, whilst preserving the spatial information necessary for fine structures. The majority of contemporary vision foundation models employ fixed patch grids, task-specific decoders, or token pruning strategies to minimise computation. However, these models generally lack a reversible account of how local patches constitute parts, objects, and scenes. The MoE-HBPE-VFM is a foundational model that incorporates the expertise of multiple specialists. The paper introduces a hierarchical visual byte-pair encoding technique for unifying multi-granularity perception, a concept that is elaborated upon in this paper. The proposed tokenizer builds a merge tree over adjacent visual tokens and stores the parent-child support of every merged token. This facilitates coarse representation expansion during dense prediction. The distribution of active hierarchical tokens to specialised visual experts is then facilitated by a sparse expert router. The processing of texture, boundary, object, scene and rare-category evidence can be undertaken in different parameter subspaces. The training goals are as follows: recognition, detection, instance segmentation, semantic parsing, promptable transfer, token compactness, routing balance and cross-scale consistency. A series of experiments were conducted on ImageNet-1 K, ImageNet-C, MS COCO, ADE20K, LVIS, Open Images V7, SA-1B and ObjectNet. The results of these experiments demonstrated that the full model enhanced the unified score by 2.2 points in comparison to the most effective dense baseline in the controlled benchmark. The findings of the ablations indicate that the combination of the hierarchical structure, sparse expert routing, load balancing, and cross-scale consistency has yielded certain enhancements. The efficiency analysis showed a favorable accuracy-computation trade-off: the model achieved a 90.8 unified score with 16.3 G active FLOPs and 31.2 ms single-image latency. However, its latency was higher than that of EfficientViT-M5, Hiera-B, and ConvNeXt V2-B; the results therefore support controlled active computation and a competitive overall trade-off rather than a universal inference-speed advantage. In consideration of the aforementioned results, it is hypothesised that a viable approach to large-scale, multi-granularity visual foundation models may be facilitated by reversible token hierarchy and sparse expert specialisation.
No takes yet. Share an insight, caveat, or question.
Pan et al. (2026) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: