Answers are grounded in this paper and cite their sources.
The rapid growth of mobile applications (particularly deployed natural language processing applications), generate a proliferation of user-generated data, encapsulating a wealth of valuable knowledge. To facilitate the analysis and extraction of insights from on-device data while adhering to stringent data privacy regulations, natural language processing (NLP) models deployed on devices are frequently finetuned within a federated learning (FL) setting. However, this process typically requires hundreds of thousands of labeled training samples from mobile users. This presents a significant challenge, as users often lack the motivation or expertise to annotate their data. The scarcity of labeled data, referred to as the few-shot scenario, poses a substantial obstacle for mobile NLP applications. This work is the first to explore federated NLP in the few-shot scenario (FedFSL). Extensive measurements are conducted to highlight that previous federated NLP suffer from significant accuracy degradation when labeled data is scarce and imbalanced distributed. By leveraging advancements in pseudolabeling and prompt learning, we establish a training pipeline that can achieve competitive accuracy on the evaluated fewshot NLP benchmarks with as few as 0.05% of training samples (fewer than 100) labeled, while the rest remain unlabeled. To make this pipeline into practice, we introduce a system, FeS1, which addresses high execution costs through three key innovations: (1) Curriculum pacing, which injects pseudo labels into the training process in alignment with the learning progress; (2) Representational diversity, a strategy that selects the most learnable data for pseudo-label generation; (3) Coplanning of the model's training depth and layer capacity. Together, these designs reduce training latency and client energy consumption by up to 46.× and 41.2×, respectively. For network traffic, FeS achieves an average reduction of 1841.7× over FedFSL and 2.17× over the stronger FedFSLBIAS baseline across four tasks. By integrating algorithmic and system-level innovations, FeS demonstrates that FL can be effectively applied in challenging scenarios where the majority of training samples are unlabeled.
No takes yet. Share an insight, caveat, or question.
Cai et al. (2026) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: