What type of study is this?

This is a Quantitative Study study.

October 3, 20250 citationsOpen Access

DiEP: Adaptive Differentiable Expert Pruning for Mixture-of-Experts Compression

DiEP: Adaptive Mixture-of-Experts Compression through Differentiable Expert Pruning

View Full Paper

Ask AI

Bookmark

Authors

Adaptive pruning improves mixture-of-experts model efficiency in NLP tasks, retaining high performance.

DiEP retains approximately 92% of performance with half the experts on Mixtral 8×7B and improves efficiency.
The proposed method outperforms existing pruning techniques by up to 7.1% on the MMLU dataset, enhancing model utility.
Non-uniform expert pruning addresses varying redundancy across layers, ensuring optimal outcome in model performance.
This approach transforms discrete search space into a continuous one, enabling effective gradient-based optimization.

Bai et al. (2025) studied this question.

View Full Paper

Ask AI

Bookmark

October 3, 20250 citationsOpen Access

DiEP: Adaptive Mixture-of-Experts Compression through Differentiable Expert Pruning

View Full Paper

Ask AI

Bookmark

Authors

Adaptive pruning improves mixture-of-experts model efficiency in NLP tasks, retaining high performance.

DiEP retains approximately 92% of performance with half the experts on Mixtral 8×7B and improves efficiency.
The proposed method outperforms existing pruning techniques by up to 7.1% on the MMLU dataset, enhancing model utility.
Non-uniform expert pruning addresses varying redundancy across layers, ensuring optimal outcome in model performance.
This approach transforms discrete search space into a continuous one, enabling effective gradient-based optimization.

Bai et al. (2025) studied this question.

View Full Paper

Ask AI

Bookmark