Phi-3.5-MoE-instruct

Phi-3.5-MoE-Instruct is a highly efficient mixture-of-experts model, featuring approximately 42 billion total parameters with 6.6 billion active parameters. Boasting an extensive 128K context window, this model demonstrates exceptional capabilities in reasoning, mathematics, coding, and multilingual applications. In numerous benchmarks, it surpasses the performance of larger, dense models. Rigorously refined through safety-focused post-training, including SFT and DPO, Phi-3.5-MoE-Instruct is released under the MIT license. It is particularly well-suited for tasks demanding both high performance and efficiency, especially in multilingual contexts or those requiring sophisticated reasoning.