Qwen2.5-Coder 7B Instruct

Qwen

Qwen2.5-Coder is a powerhouse coding model, expertly trained on 5.5 trillion code tokens and fluent in 92 programming languages. Equipped with a 128K context window, it's engineered for top-tier code generation, intelligent completion, and precise repair. Beyond coding, it retains impressive skills in mathematics and general problem-solving. This model truly shines in complex, multi-language projects and demonstrates advanced code reasoning abilities.

Model Specifications

Technical details and capabilities of Qwen2.5-Coder 7B Instruct

Core Specifications

7.0B Parameters

Model size and complexity

5500.0B Training Tokens

Amount of data used in training

128.0K / 128.0K

Input / Output tokens

February 29, 2024

Knowledge cutoff date

September 18, 2024

Release date

Capabilities & License

Multimodal Support

Not Supported

Web Hydrated

No

License

Apache 2.0

Resources

Research Paper

https://arxiv.org/abs/2409.12186

API Reference

https://www.alibabacloud.com/help/en/model-studio/developer-reference/use-qwen-by-calling-api

Code Repository

https://github.com/QwenLM/Qwen2

Performance Insights

Check out how Qwen2.5-Coder 7B Instruct handles various AI tasks through comprehensive benchmark results.

90

68

45

23

0

88.4

HumanEval

88.4

(98%)

83.9

GSM8K

83.9

(93%)

83.5

MBPP

83.5

(93%)

76.8

HellaSwag

76.8

(85%)

72.9

WinoGrande

72.9

(81%)

68

MMLU-Base

68

(76%)

67.6

MMLU

67.6

(75%)

66.6

MMLU-Redux

66.6

(74%)

60.9

ARC-Challenge

60.9

(68%)

56.5

CRUXEval-Input-CoT

56.5

(63%)

56.0

CRUXEval-Output-CoT

56.0

(62%)

55.6

Aider

55.6

(62%)

50.6

TruthfulQA

50.6

(56%)

46.6

MATH

46.6

(52%)

41

BigCodeBench

41

(46%)

40.1

MMLU-Pro

40.1

(45%)

34

STEM

34

(38%)

34

TheoremQA

34

(38%)

18.2

LiveCodeBench

18.2

(20%)

HumanEval

GSM8K

MBPP

HellaSwag

WinoGrande

MMLU-Base

MMLU

MMLU-Redux

ARC-Challenge

CRUXEval-Input-CoT

CRUXEval-Output-CoT

Aider

TruthfulQA

MATH

BigCodeBench

MMLU-Pro

STEM

TheoremQA

LiveCodeBench

Detailed Benchmarks

Dive deeper into Qwen2.5-Coder 7B Instruct's performance across specific task categories. Expand each section to see detailed metrics and comparisons.

Math

GSM8K

97.1%

88.9%

Phi-3.5-MoE-instruct

88.7%

Gemini 1.5 Flash

86.2%

Phi-3.5-mini-instruct

86.2%

Qwen2.5-Coder 7B Instruct

83.9%

Qwen2 7B Instruct

82.3%

Llama 3.2 3B Instruct

77.7%

74.0%

68.6%

Current model

Other models

Avg (83.4%)

Coding

HumanEval

Claude 3.5 Sonnet

93.7%

Llama 3.1 405B Instruct

89.0%

Mistral Small 3.1 24B

88.4%

Llama 3.3 70B Instruct

88.4%

Qwen2.5 32B Instruct

88.4%

Qwen2.5-Coder 7B Instruct

88.4%

88.4%

Claude 3.5 Haiku

88.1%

88.1%

Ministral 8B Instruct

34.8%

Current model

Other models

Avg (83.6%)

MBPP

Qwen2.5-Coder 32B Instruct

90.2%

Qwen2.5 72B Instruct

88.2%

Qwen2.5 32B Instruct

84.0%

Qwen2.5-Coder 7B Instruct

83.5%

Qwen2.5 14B Instruct

82.0%

Phi-3.5-MoE-instruct

80.8%

Qwen2 72B Instruct

80.2%

Qwen2.5 7B Instruct

79.2%

52.4%

Current model

Other models

Avg (80.1%)

LiveCodeBench

40.5%

39.0%

37.6%

Gemini 2.0 Flash

35.1%

Qwen2.5-Coder 32B Instruct

31.4%

Qwen2.5 7B Instruct

28.7%

Qwen2 7B Instruct

26.6%

Qwen2.5-Coder 7B Instruct

18.2%

Current model

Other models

Avg (32.1%)

Reasoning

HellaSwag

95.4%

Phi-3.5-MoE-instruct

83.8%

Mistral NeMo Instruct

83.5%

Qwen2.5-Coder 32B Instruct

83.0%

81.9%

Qwen2.5-Coder 7B Instruct

76.8%

Llama 3.2 3B Instruct

69.8%

Phi-3.5-mini-instruct

69.4%

Current model

Other models

Avg (80.4%)

Knowledge

MATH

97.9%

49.7%

Qwen2 7B Instruct

49.6%

Phi-3.5-mini-instruct

48.5%

Llama 3.2 3B Instruct

48.0%

Qwen2.5-Coder 7B Instruct

46.6%

Claude 3 Sonnet

43.1%

43.1%

42.3%

32.6%

Current model

Other models

Avg (50.1%)

MMLU

91.8%

Llama 3.1 8B Instruct

69.4%

69.2%

Phi-3.5-mini-instruct

69.0%

Mistral NeMo Instruct

68.0%

Qwen2.5-Coder 7B Instruct

67.6%

Ministral 8B Instruct

65.0%

Llama 3.2 3B Instruct

63.4%

Current model

Other models

Avg (70.4%)

Hallucination

TruthfulQA

Qwen2.5 14B Instruct

58.4%

Jamba 1.5 Large

58.3%

Qwen2.5 32B Instruct

57.8%

Qwen2 72B Instruct

54.8%

Qwen2.5-Coder 32B Instruct

54.2%

54.1%

Qwen2.5-Coder 7B Instruct

50.6%

Mistral NeMo Instruct

50.3%

Current model

Other models

Avg (54.8%)

Non categorized

TheoremQA

Qwen2.5 32B Instruct

44.1%

Qwen2.5-Coder 32B Instruct

43.1%

Qwen2.5 14B Instruct

43.0%

Qwen2.5-Coder 7B Instruct

34.0%

Current model

Other models

Avg (41.1%)

MMLU-Pro

Jamba 1.5 Large

53.5%

51.0%

Qwen2.5-Coder 32B Instruct

50.4%

Llama 3.1 8B Instruct

48.3%

Phi-3.5-mini-instruct

47.4%

Qwen2 7B Instruct

44.1%

42.5%

Qwen2.5-Coder 7B Instruct

40.1%

Current model

Other models

Avg (47.2%)

MMLU-Redux

92.9%

89.1%

Qwen2.5 72B Instruct

86.8%

Qwen2.5 32B Instruct

83.9%

Qwen2.5 14B Instruct

80.0%

Qwen2.5-Coder 32B Instruct

77.5%

Qwen2.5 7B Instruct

75.4%

Qwen2.5-Coder 7B Instruct

66.6%

Current model

Other models

Avg (81.5%)

ARC-Challenge

Qwen2.5-Coder 32B Instruct

70.5%

Qwen2.5-Coder 7B Instruct

60.9%

Current model

Other models

Avg (65.7%)

WinoGrande

83.7%

Phi-3.5-MoE-instruct

81.3%

Qwen2.5-Coder 32B Instruct

80.8%

Qwen2.5-Coder 7B Instruct

72.9%

Phi-3.5-mini-instruct

68.5%

Current model

Other models

Avg (77.4%)

Aider

72.2%

Qwen2.5-Coder 7B Instruct

55.6%

Current model

Other models

Avg (63.9%)

Providers Pricing Coming Soon

We're working on gathering comprehensive pricing data from all major providers for Qwen2.5-Coder 7B Instruct. Compare costs across platforms to find the best pricing for your use case.

OpenAI

Anthropic

Google

Mistral AI

Cohere

Share your feedback

Hi, I'm Charlie Palars, the founder of Deepranking.ai. I'm always looking for ways to improve the site and make it more useful for you. You can write me through this form or directly through X at @palarsio.

Your feedback helps us improve our service

Stay Ahead with AI Updates

Get insights on Gemini Pro 2.5, Sonnet 3.7 and more top AI models