Llama 3.1 405B Instruct

Meta Llama

Llama 3.1 405B Instruct is a powerful language model designed for engaging in multilingual conversations. It excels in multiple languages and surpasses the performance of numerous open-source and proprietary chat models on standard industry evaluations. With support for 8 languages and an extensive 128K token context window, it's built for complex and nuanced dialogue.

Model Specifications

Technical details and capabilities of Llama 3.1 405B Instruct

Core Specifications

405.0B Parameters

Model size and complexity

15000.0B Training Tokens

Amount of data used in training

128.0K / 128.0K

Input / Output tokens

November 30, 2023

Knowledge cutoff date

July 22, 2024

Release date

Capabilities & License

Multimodal Support

Not Supported

Web Hydrated

No

License

Llama 3.1 Community License

Resources

API Reference

https://github.com/meta-llama/llama-models

Playground

https://llama.meta.com/llama-downloads

Code Repository

https://github.com/meta-llama/llama-models

Performance Insights

Check out how Llama 3.1 405B Instruct handles various AI tasks through comprehensive benchmark results.

100

75

50

25

0

96.9

ARC Challenge

96.9

(97%)

96.8

GSM8K

96.8

(97%)

92

API-Bank

92

(92%)

91.6

Multilingual MGSM

91.6

(92%)

89

HumanEval

89

(89%)

88.6

MMLU (CoT)

88.6

(89%)

88.6

IFEval

88.6

(89%)

88.6

MBPP EvalPlus

88.6

(89%)

88.5

BFCL

88.5

(89%)

87.3

MMLU

87.3

(87%)

84.8

DROP

84.8

(85%)

75.2

Multipl-E HumanEval

75.2

(75%)

73.8

MATH

73.8

(74%)

73.3

MMLU-Pro

73.3

(73%)

65.7

Multipl-E MBPP

65.7

(66%)

58.7

Nexus

58.7

(59%)

50.7

GPQA

50.7

(51%)

35.3

Gorilla Benchmark API Bench

35.3

(35%)

ARC Challenge

GSM8K

API-Bank

Multilingual MGSM

HumanEval

MMLU (CoT)

IFEval

MBPP EvalPlus

BFCL

MMLU

DROP

Multipl-E HumanEval

MATH

MMLU-Pro

Multipl-E MBPP

Nexus

GPQA

Gorilla Benchmark API Bench

Model Comparison

See how Llama 3.1 405B Instruct stacks up against other leading models across key performance metrics.

100

80

60

40

20

0

87.3

MMLU - Llama 3.1 405B Instruct

87.3

(87%)

86.2

MMLU - Grok-2 mini

86.2

(86%)

88.7

MMLU - GPT-4o

88.7

(89%)

86

MMLU - Llama 3.3 70B Instruct

86

(86%)

87.5

MMLU - Grok-2

87.5

(88%)

90.4

MMLU - Claude 3.5 Sonnet

90.4

(90%)

73.3

MMLU-Pro - Llama 3.1 405B Instruct

73.3

(73%)

72

MMLU-Pro - Grok-2 mini

72

(72%)

72.6

MMLU-Pro - GPT-4o

72.6

(73%)

68.9

MMLU-Pro - Llama 3.3 70B Instruct

68.9

(69%)

75.5

MMLU-Pro - Grok-2

75.5

(76%)

76.1

MMLU-Pro - Claude 3.5 Sonnet

76.1

(76%)

50.7

GPQA - Llama 3.1 405B Instruct

50.7

(51%)

51

GPQA - Grok-2 mini

51

(51%)

53.6

GPQA - GPT-4o

53.6

(54%)

50.5

GPQA - Llama 3.3 70B Instruct

50.5

(51%)

56.0

GPQA - Grok-2

56.0

(56%)

59.4

GPQA - Claude 3.5 Sonnet

59.4

(59%)

89

HumanEval - Llama 3.1 405B Instruct

89

(89%)

85.7

HumanEval - Grok-2 mini

85.7

(86%)

90.2

HumanEval - GPT-4o

90.2

(90%)

88.4

HumanEval - Llama 3.3 70B Instruct

88.4

(88%)

88.4

HumanEval - Grok-2

88.4

(88%)

92

HumanEval - Claude 3.5 Sonnet

92

(92%)

73.8

MATH - Llama 3.1 405B Instruct

73.8

(74%)

73

MATH - Grok-2 mini

73

(73%)

76.6

MATH - GPT-4o

76.6

(77%)

77

MATH - Llama 3.3 70B Instruct

77

(77%)

76.1

MATH - Grok-2

76.1

(76%)

71.1

MATH - Claude 3.5 Sonnet

71.1

(71%)

MMLU

MMLU-Pro

GPQA

HumanEval

MATH

Llama 3.1 405B Instruct

Grok-2 mini

GPT-4o

Llama 3.3 70B Instruct

Grok-2

Claude 3.5 Sonnet

Detailed Benchmarks

Dive deeper into Llama 3.1 405B Instruct's performance across specific task categories. Expand each section to see detailed metrics and comparisons.

Math

GSM8K

97.1%

Llama 3.1 405B Instruct

96.8%

Claude 3.5 Sonnet

96.4%

Claude 3.5 Sonnet

96.4%

Qwen2.5 32B Instruct

95.9%

95.9%

Qwen2.5 72B Instruct

95.8%

95.1%

Current model

Other models

Avg (96.2%)

Coding

HumanEval

Claude 3.5 Sonnet

93.7%

Mistral Large 2

92.0%

90.2%

89.0%

89.0%

Llama 3.1 405B Instruct

89.0%

Mistral Small 3.1 24B

88.4%

Llama 3.3 70B Instruct

88.4%

Qwen2.5 32B Instruct

88.4%

Ministral 8B Instruct

34.8%

Current model

Other models

Avg (84.3%)

Reasoning

DROP

92.2%

Claude 3.5 Sonnet

87.1%

Claude 3.5 Sonnet

87.1%

86.0%

85.4%

Llama 3.1 405B Instruct

84.8%

83.4%

Claude 3.5 Haiku

83.1%

83.1%

Llama 3.1 8B Instruct

59.5%

Current model

Other models

Avg (83.2%)

Knowledge

MMLU

91.8%

88.5%

88.0%

87.5%

87.4%

Llama 3.1 405B Instruct

87.3%

86.9%

86.8%

86.5%

Llama 3.2 3B Instruct

63.4%

Current model

Other models

Avg (85.4%)

GPQA

87.7%

53.6%

53.6%

Gemini 1.5 Flash

51.0%

51.0%

Llama 3.1 405B Instruct

50.7%

Llama 3.3 70B Instruct

50.5%

50.4%

Qwen2.5 32B Instruct

49.5%

Qwen2 7B Instruct

25.3%

Current model

Other models

Avg (52.3%)

MATH

97.9%

76.6%

76.1%

Qwen2.5 7B Instruct

75.5%

74.7%

Llama 3.1 405B Instruct

73.8%

73.3%

73.0%

72.6%

32.6%

Current model

Other models

Avg (72.6%)

Non categorized

MMLU (CoT)

Llama 3.1 405B Instruct

88.6%

Llama 3.1 70B Instruct

86.0%

Llama 3.1 8B Instruct

73.0%

Current model

Other models

Avg (82.5%)

MMLU-Pro

84.0%

75.9%

75.8%

75.5%

74.7%

Llama 3.1 405B Instruct

73.3%

72.6%

72.0%

Qwen2.5 72B Instruct

71.1%

Qwen2.5-Coder 7B Instruct

40.1%

Current model

Other models

Avg (71.5%)

IFEval

Claude 3.7 Sonnet

93.2%

Llama 3.3 70B Instruct

92.1%

90.4%

90.0%

89.7%

Llama 3.1 405B Instruct

88.6%

Llama 3.1 70B Instruct

87.5%

87.2%

86.1%

61.3%

Current model

Other models

Avg (86.6%)

ARC Challenge

Llama 3.1 405B Instruct

96.9%

96.4%

Llama 3.1 70B Instruct

94.8%

Claude 3 Sonnet

93.2%

Jamba 1.5 Large

93.0%

Phi-3.5-MoE-instruct

91.0%

89.2%

85.7%

Current model

Other models

Avg (92.5%)

MBPP EvalPlus

Llama 3.1 405B Instruct

88.6%

Llama 3.3 70B Instruct

87.6%

Current model

Other models

Avg (88.1%)

Multipl-E HumanEval

Llama 3.1 405B Instruct

75.2%

Llama 3.1 70B Instruct

65.5%

Llama 3.1 8B Instruct

50.8%

Current model

Other models

Avg (63.8%)

Multipl-E MBPP

Llama 3.1 405B Instruct

65.7%

Llama 3.1 70B Instruct

62.0%

Llama 3.1 8B Instruct

52.4%

Current model

Other models

Avg (60.0%)

API-Bank

Llama 3.1 405B Instruct

92.0%

Llama 3.1 70B Instruct

90.0%

Llama 3.1 8B Instruct

82.6%

Current model

Other models

Avg (88.2%)

BFCL

Llama 3.1 405B Instruct

88.5%

Llama 3.1 70B Instruct

84.8%

Llama 3.1 8B Instruct

76.1%

74.0%

68.4%

66.6%

66.4%

65.0%

Current model

Other models

Avg (73.7%)

Gorilla Benchmark API Bench

Llama 3.1 405B Instruct

35.3%

Llama 3.1 70B Instruct

29.7%

Llama 3.1 8B Instruct

8.2%

Current model

Other models

Avg (24.4%)

Nexus

Llama 3.1 405B Instruct

58.7%

Llama 3.1 70B Instruct

56.7%

Llama 3.1 8B Instruct

38.5%

Llama 3.2 3B Instruct

34.3%

Current model

Other models

Avg (47.0%)

Providers Pricing Coming Soon

We're working on gathering comprehensive pricing data from all major providers for Llama 3.1 405B Instruct. Compare costs across platforms to find the best pricing for your use case.

OpenAI

Anthropic

Google

Mistral AI

Cohere

Share your feedback

Hi, I'm Charlie Palars, the founder of Deepranking.ai. I'm always looking for ways to improve the site and make it more useful for you. You can write me through this form or directly through X at @palarsio.

Your feedback helps us improve our service

Stay Ahead with AI Updates

Get insights on Gemini Pro 2.5, Sonnet 3.7 and more top AI models