What question did this study set out to answer?

The research aims to compress verbose tool schemas using advanced attention mechanisms to enhance tool use in language models.

March 1, 2026Open Access

ToolFormerMicro: Composable Tool Schema Compression via Gated Cross-Attention

Key Points

The research aims to compress verbose tool schemas using advanced attention mechanisms to enhance tool use in language models.
Developed a 428M parameter encoder-decoder model called ToolFormerMicro.
Implemented gated cross-attention to transform tool schemas into 8-token gist vectors.
Utilized a three-stage curriculum involving schema auto-encoding, contrastive discrimination, and end-to-end tool calling.
Achieved 0.818 Tool Selection Accuracy with zero false positives across different tool scenarios.
Demonstrated order independence in tool composability with a delta of 0.000.
Showcased sub-linear scaling with up to 200 tools and effective cache invalidation on hot-swap.

Abstract

ToolFormerMicro is a ~428M parameter encoder-decoder model that compresses verbose tool schemas into compact 8-token gist vectors via gated cross-attention. Initialized from Qwen2.5-0.5B and trained with a three-stage curriculum (schema auto-encoding, contrastive discrimination, end-to-end tool calling), the model achieves 0.818 Tool Selection Accuracy with zero false positives across seen, held-out, and unseen tool splits. Three composability experiments verify order independence (delta=0.000), sub-linear scaling to 200 tools, and bit-identical cache invalidation on hot-swap — properties that make gist representations a drop-in composable primitive for tool-augmented LLM serving.

Read Full Paperexternally

Mark Helpful

Bookmark

Relay

View Full Paper