Grok-1.5

xAI

This is a cutting-edge language model that demonstrates superior reasoning skills, especially in coding and mathematical domains. It boasts a 128K token context window, allowing for more comprehensive analysis and significantly improved problem-solving compared to previous models.

Model Specifications

Technical details and capabilities of Grok-1.5

Core Specifications

128.0K / 128.0K

Input / Output tokens

March 27, 2024

Release date

Capabilities & License

Multimodal Support

Not Supported

Web Hydrated

License

Proprietary

Resources

API Reference

https://x.ai/api

Code Repository

https://github.com/xai-org/grok-1

Performance Insights

Check out how Grok-1.5 handles various AI tasks through comprehensive benchmark results.

GSM8K

(100%)

85.6

DocVQA

85.6

(95%)

81.3

MMLU

81.3

(90%)

74.1

HumanEval

74.1

(82%)

53.6

MMMU

53.6

(60%)

52.8

MathVista

52.8

(59%)

MMLU-Pro

(57%)

50.6

MATH

50.6

(56%)

35.9

GPQA

35.9

(40%)

GSM8K

DocVQA

MMLU

HumanEval

MMMU

MathVista

MMLU-Pro

MATH

GPQA

Detailed Benchmarks

Dive deeper into Grok-1.5's performance across specific task categories. Expand each section to see detailed metrics and comparisons.

Math

GSM8K

97.1%

Llama 3.1 Nemotron 70B Instruct

91.4%

Qwen2.5-Coder 32B Instruct

91.1%

91.1%

90.8%

90.0%

88.9%

88.7%

86.2%

68.6%

Current model

Other models

Avg (88.4%)

Coding

HumanEval

Claude 3.5 Sonnet

93.7%

Codestral-22B

81.1%

Llama 3.1 70B Instruct

80.5%

75.9%

74.3%

74.1%

73.0%

Llama 3.1 8B Instruct

72.6%

Pixtral-12B

72.0%

Ministral 8B Instruct

34.8%

Current model

Other models

Avg (73.2%)

Knowledge

GPQA

87.7%

38.4%

36.9%

36.8%

36.4%

35.9%

35.7%

33.3%

Llama 3.2 11B Instruct

32.8%

Qwen2 7B Instruct

25.3%

Current model

Other models

Avg (39.9%)

MMLU

91.8%

Llama 3.1 70B Instruct

83.6%

83.3%

82.3%

82.0%

81.3%

81.2%

Mistral Small 3.1 24B

80.6%

Nova Lite

80.5%

Llama 3.2 3B Instruct

63.4%

Current model

Other models

Avg (81.0%)

MATH

o3-mini

97.9%

Gemini 1.5 Flash 8B

58.7%

Qwen2.5-Coder 32B Instruct

57.2%

Ministral 8B Instruct

54.5%

Llama 3.2 11B Instruct

51.9%

Grok-1.5

50.6%

Olmo 2 32B

49.7%

Qwen2 7B Instruct

49.6%

Phi-3.5-mini-instruct

48.5%

Gemini 1.0 Pro

32.6%

Current model

Other models

Avg (55.1%)

Non categorized

MMLU-Pro

84.0%

56.8%

56.3%

54.3%

53.5%

51.0%

Qwen2.5-Coder 32B Instruct

50.4%

Llama 3.1 8B Instruct

48.3%

Phi-3.5-mini-instruct

47.4%

Qwen2.5-Coder 7B Instruct

40.1%

Current model

Other models

Avg (54.2%)

MMMU

Gemini Pro 2.5 Experimental

81.7%

Llama 3.2 90B Instruct

60.3%

59.4%

56.2%

53.7%

53.6%

53.6%

52.5%

Llama 3.2 11B Instruct

50.7%

GPT-3.5 Turbo

0.0%

Current model

Other models

Avg (52.2%)

MathVista

Kimi-k1.5

74.9%

GPT-4o

63.8%

Llama 3.2 90B Instruct

57.3%

GPT-4o mini

56.7%

Gemini 1.5 Flash 8B

54.7%

Grok-1.5

52.8%

Llama 3.2 11B Instruct

51.5%

Gemini 1.0 Pro

46.6%

Phi-3.5-vision-instruct

43.9%

GPT-3.5 Turbo

0.0%

Current model

Other models

Avg (50.2%)

DocVQA

93.3%

93.2%

92.8%

90.7%

Llama 3.2 90B Instruct

90.1%

Llama 3.2 11B Instruct

88.4%

Grok-1.5

85.6%

Grok-1.5V

85.6%

Current model

Other models

Avg (90.0%)

Providers Pricing Coming Soon

We're working on gathering comprehensive pricing data from all major providers for Grok-1.5. Compare costs across platforms to find the best pricing for your use case.

OpenAI

Anthropic

Google

Mistral AI

Cohere

Share your feedback

Hi, I'm Charlie Palars, the founder of Deepranking.ai. I'm always looking for ways to improve the site and make it more useful for you. You can write me through this form or directly through X at @palarsio.

Your feedback helps us improve our service

Grok-1.5

Model Specifications

Core Specifications

Capabilities & License

Resources

Performance Insights

Detailed Benchmarks

Math

GSM8K

Coding

HumanEval

Knowledge

GPQA

MMLU

MATH

Non categorized

MMLU-Pro

MMMU

MathVista

DocVQA

Providers Pricing Coming Soon

Share your feedback

Stay Ahead with AI Updates