Claude 3 Haiku

Anthropic

Claude 3 Haiku stands out as the quickest and most streamlined model within the Claude 3 series, engineered for rapid-fire responses. It's exceptionally good at providing fast answers to straightforward questions and requests. This makes it perfect for creating smooth AI interactions that feel natural and human-like.

Model Specifications

Technical details and capabilities of Claude 3 Haiku

Core Specifications

200.0K / 200.0K

Input / Output tokens

March 12, 2024

Release date

Capabilities & License

Multimodal Support

Supported

Web Hydrated

License

Proprietary

Resources

Research Paper

https://www-cdn.anthropic.com/de8ba9b01c9ab7cbabf5c33b80b7bbc618857627/Model_Card_Claude_3.pdf

API Reference

https://www.anthropic.com/claude

Playground

https://claude.ai

Performance Insights

Check out how Claude 3 Haiku handles various AI tasks through comprehensive benchmark results.

89.2

ARC Challenge

89.2

(99%)

88.9

GSM8K

88.9

(99%)

85.9

HellaSwag

85.9

(95%)

78.4

DROP

78.4

(87%)

75.9

HumanEval

75.9

(84%)

75.2

MMLU

75.2

(84%)

75.1

MGSM

75.1

(83%)

73.7

BIG-Bench-Hard

73.7

(82%)

38.9

MATH

38.9

(43%)

33.3

GPQA

33.3

(37%)

ARC Challenge

GSM8K

HellaSwag

DROP

HumanEval

MMLU

MGSM

BIG-Bench-Hard

MATH

GPQA

Model Comparison

See how Claude 3 Haiku stacks up against other leading models across key performance metrics.

100

75.2

MMLU - Claude 3 Haiku

75.2

(75%)

MMLU - Claude 3 Sonnet

(79%)

81.3

MMLU - Grok-1.5

81.3

(81%)

MMLU - Phi-3.5-mini-instruct

(69%)

78.9

MMLU - Phi-3.5-MoE-instruct

78.9

(79%)

82.3

MMLU - Qwen2 72B Instruct

82.3

(82%)

33.3

GPQA - Claude 3 Haiku

33.3

(33%)

40.4

GPQA - Claude 3 Sonnet

40.4

(40%)

35.9

GPQA - Grok-1.5

35.9

(36%)

30.4

GPQA - Phi-3.5-mini-instruct

30.4

(30%)

36.8

GPQA - Phi-3.5-MoE-instruct

36.8

(37%)

42.4

GPQA - Qwen2 72B Instruct

42.4

(42%)

88.9

GSM8K - Claude 3 Haiku

88.9

(89%)

92.3

GSM8K - Claude 3 Sonnet

92.3

(92%)

GSM8K - Grok-1.5

(90%)

86.2

GSM8K - Phi-3.5-mini-instruct

86.2

(86%)

88.7

GSM8K - Phi-3.5-MoE-instruct

88.7

(89%)

91.1

GSM8K - Qwen2 72B Instruct

91.1

(91%)

38.9

MATH - Claude 3 Haiku

38.9

(39%)

43.1

MATH - Claude 3 Sonnet

43.1

(43%)

50.6

MATH - Grok-1.5

50.6

(51%)

48.5

MATH - Phi-3.5-mini-instruct

48.5

(49%)

59.5

MATH - Phi-3.5-MoE-instruct

59.5

(60%)

59.7

MATH - Qwen2 72B Instruct

59.7

(60%)

75.9

HumanEval - Claude 3 Haiku

75.9

(76%)

HumanEval - Claude 3 Sonnet

(73%)

74.1

HumanEval - Grok-1.5

74.1

(74%)

62.8

HumanEval - Phi-3.5-mini-instruct

62.8

(63%)

70.7

HumanEval - Phi-3.5-MoE-instruct

70.7

(71%)

HumanEval - Qwen2 72B Instruct

(86%)

MMLU

GPQA

GSM8K

MATH

HumanEval

Claude 3 Haiku

Claude 3 Sonnet

Grok-1.5

Phi-3.5-mini-instruct

Phi-3.5-MoE-instruct

Qwen2 72B Instruct

Detailed Benchmarks

Dive deeper into Claude 3 Haiku's performance across specific task categories. Expand each section to see detailed metrics and comparisons.

Math

GSM8K

97.1%

Qwen2.5-Coder 32B Instruct

91.1%

91.1%

90.8%

90.0%

88.9%

88.7%

86.2%

Phi-3.5-mini-instruct

86.2%

Gemma 2 9B

68.6%

Current model

Other models

Avg (87.9%)

Coding

HumanEval

93.7%

82.6%

81.1%

81.1%

Llama 3.1 70B Instruct

80.5%

75.9%

74.3%

74.1%

73.0%

Ministral 8B Instruct

34.8%

Current model

Other models

Avg (75.1%)

Reasoning

DROP

DeepSeek-R1

92.2%

GPT-4o mini

79.7%

Llama 3.1 70B Instruct

79.6%

79.3%

78.9%

78.4%

75.5%

74.9%

74.3%

Llama 3.1 8B Instruct

59.5%

Current model

Other models

Avg (77.2%)

HellaSwag

95.4%

89.0%

88.6%

87.6%

86.4%

85.9%

85.2%

83.8%

Mistral NeMo Instruct

83.5%

Phi-3.5-mini-instruct

69.4%

Current model

Other models

Avg (85.5%)

Knowledge

MMLU

91.8%

78.9%

77.6%

76.9%

75.7%

75.2%

75.2%

Qwen2.5-Coder 32B Instruct

75.1%

Olmo 2 32B

74.9%

Llama 3.2 3B Instruct

63.4%

Current model

Other models

Avg (76.5%)

GPQA

87.7%

36.8%

36.4%

35.9%

35.7%

33.3%

Llama 3.2 11B Instruct

32.8%

Llama 3.2 3B Instruct

32.8%

Jamba 1.5 Mini

32.3%

Qwen2 7B Instruct

25.3%

Current model

Other models

Avg (38.9%)

MATH

97.9%

43.1%

43.1%

42.3%

42.0%

38.9%

36.6%

32.6%

Current model

Other models

Avg (47.1%)

Non categorized

MGSM

92.0%

85.6%

83.5%

82.6%

80.6%

75.1%

74.5%

Llama 3.2 11B Instruct

68.9%

Phi-3.5-MoE-instruct

58.7%

Phi-3.5-mini-instruct

47.9%

Current model

Other models

Avg (74.9%)

BIG-Bench-Hard

Claude 3 Opus

86.8%

Claude 3 Sonnet

82.9%

Claude 3 Haiku

73.7%

Current model

Other models

Avg (81.1%)

ARC Challenge

Llama 3.1 405B Instruct

96.9%

Llama 3.1 70B Instruct

94.8%

93.2%

93.0%

91.0%

89.2%

85.7%

Phi-3.5-mini-instruct

84.6%

Llama 3.1 8B Instruct

83.4%

Qwen2 72B Instruct

68.9%

Current model

Other models

Avg (88.1%)

Providers Pricing Coming Soon

We're working on gathering comprehensive pricing data from all major providers for Claude 3 Haiku. Compare costs across platforms to find the best pricing for your use case.

OpenAI

Anthropic

Google

Mistral AI

Cohere

Share your feedback

Hi, I'm Charlie Palars, the founder of Deepranking.ai. I'm always looking for ways to improve the site and make it more useful for you. You can write me through this form or directly through X at @palarsio.

Your feedback helps us improve our service

Claude 3 Haiku

Model Specifications

Core Specifications

Capabilities & License

Resources

Performance Insights

Model Comparison

Detailed Benchmarks

Math

GSM8K

Coding

HumanEval

Reasoning

DROP

HellaSwag

Knowledge

MMLU

GPQA

MATH

Non categorized

MGSM

BIG-Bench-Hard

ARC Challenge

Providers Pricing Coming Soon

Share your feedback

Stay Ahead with AI Updates