Gemini 1.5 Pro

Google

Gemini 1.5 Pro is a versatile, mid-size multimodal AI model designed for advanced reasoning across diverse tasks. Its standout feature is the ability to handle massive datasets in a single pass. For example, it can analyze two-hour videos, 19-hour audio files, code repositories containing 60,000 lines, or documents as large as 2,000 pages.

Model Specifications

Technical details and capabilities of Gemini 1.5 Pro

Core Specifications

2.1M / 8.2K

Input / Output tokens

October 31, 2023

Knowledge cutoff date

April 30, 2024

Release date

Capabilities & License

Multimodal Support

Supported

Web Hydrated

License

Proprietary

Resources

Research Paper

https://arxiv.org/pdf/2403.05530

API Reference

https://ai.google.dev/gemini-api/docs/models/gemini#gemini-1.5-pro

Playground

https://ai.google.dev/studio

Performance Insights

Check out how Gemini 1.5 Pro handles various AI tasks through comprehensive benchmark results.

100

98.8

XSTest

98.8

(99%)

93.3

Hellaswag

93.3

(93%)

90.8

GSM8K

90.8

(91%)

89.2

BigBench-Hard

89.2

(89%)

87.5

MGSM

87.5

(88%)

86.5

MATH

86.5

(87%)

85.9

MMLU

85.9

(86%)

85.4

Natural2Code

85.4

(85%)

84.1

HumanEval

84.1

(84%)

82.6

MRCR

82.6

(83%)

78.6

Video-MME

78.6

(79%)

75.8

MMLU-Pro

75.8

(76%)

75.1

WMT23

75.1

(75%)

74.9

DROP

74.9

(75%)

68.1

MathVista

68.1

(68%)

65.9

MMMU

65.9

(66%)

64.6

FunctionalMATH

64.6

(65%)

63.9

PhysicsFinals

63.9

(64%)

59.1

GPQA

59.1

(59%)

53.9

Vibe-Eval

53.9

(54%)

HiddenMath

(52%)

46.4

AMC_2022_23

46.4

(46%)

6.7

FLEURS

6.7

(7%)

XSTest

Hellaswag

GSM8K

BigBench-Hard

MGSM

MATH

MMLU

Natural2Code

HumanEval

MRCR

Video-MME

MMLU-Pro

WMT23

DROP

MathVista

MMMU

FunctionalMATH

PhysicsFinals

GPQA

Vibe-Eval

HiddenMath

AMC_2022_23

FLEURS

Detailed Benchmarks

Dive deeper into Gemini 1.5 Pro's performance across specific task categories. Expand each section to see detailed metrics and comparisons.

Math

GSM8K

97.1%

Qwen2.5 7B Instruct

91.6%

Llama 3.1 Nemotron 70B Instruct

91.4%

Qwen2.5-Coder 32B Instruct

91.1%

91.1%

90.8%

90.0%

88.9%

88.7%

68.6%

Current model

Other models

Avg (88.9%)

Coding

HumanEval

93.7%

85.4%

84.9%

84.8%

84.8%

84.1%

83.5%

82.6%

81.1%

Ministral 8B Instruct

34.8%

Current model

Other models

Avg (80.0%)

Reasoning

DROP

92.2%

79.3%

78.9%

78.4%

75.5%

74.9%

74.3%

70.2%

Llama 3.1 8B Instruct

59.5%

Current model

Other models

Avg (75.9%)

Hellaswag

Gemini 1.5 Pro

93.3%

Gemini 1.5 Flash

86.5%

Current model

Other models

Avg (89.9%)

Knowledge

MMLU

91.8%

Grok-2 mini

86.2%

Llama 3.2 90B Instruct

86.0%

Llama 3.3 70B Instruct

86.0%

85.9%

85.9%

85.2%

84.8%

84.0%

Llama 3.2 3B Instruct

63.4%

Current model

Other models

Avg (83.9%)

MATH

o3-mini

97.9%

96.4%

89.7%

89.0%

86.5%

85.5%

83.1%

83.1%

32.6%

Current model

Other models

Avg (82.6%)

GPQA

87.7%

62.1%

60.0%

59.4%

59.1%

59.1%

56.1%

56.0%

53.6%

25.3%

Current model

Other models

Avg (57.8%)

Non categorized

MMLU-Pro

84.0%

77.6%

76.4%

76.1%

75.9%

75.8%

75.5%

74.7%

Llama 3.1 405B Instruct

73.3%

Qwen2.5-Coder 7B Instruct

40.1%

Current model

Other models

Avg (72.9%)

PhysicsFinals

Gemini 1.5 Pro

63.9%

Gemini 1.5 Flash

57.4%

Current model

Other models

Avg (60.7%)

AMC_2022_23

Gemini 1.5 Pro

46.4%

Gemini 1.5 Flash

34.8%

Current model

Other models

Avg (40.6%)

MGSM

o3-mini

92.0%

Claude 3 Opus

90.7%

GPT-4o

90.5%

89.3%

GPT-4 Turbo

88.5%

Gemini 1.5 Pro

87.5%

GPT-4o mini

87.0%

Llama 3.2 90B Instruct

86.9%

Claude 3.5 Haiku

85.6%

Phi-3.5-mini-instruct

47.9%

Current model

Other models

Avg (84.6%)

Natural2Code

92.9%

85.4%

79.8%

75.5%

Current model

Other models

Avg (83.4%)

HiddenMath

63.0%

56.0%

52.0%

47.2%

32.8%

Current model

Other models

Avg (50.2%)

MRCR

Gemini Pro 2.5 Experimental

83.1%

82.6%

71.9%

69.2%

54.7%

48.8%

36.3%

Current model

Other models

Avg (63.8%)

WMT23

75.1%

74.1%

72.6%

71.7%

Current model

Other models

Avg (73.4%)

MMMU

Gemini Pro 2.5 Experimental

81.7%

70.0%

69.1%

68.3%

66.1%

65.9%

64.0%

63.2%

Mistral Small 3.1 24B

62.8%

GPT-3.5 Turbo

0.0%

Current model

Other models

Avg (61.1%)

MathVista

Kimi-k1.5

74.9%

71.0%

Pixtral Large

69.4%

Grok-2

69.0%

Mistral Small 3.1 24B

68.9%

68.1%

68.1%

67.7%

65.8%

0.0%

Current model

Other models

Avg (62.3%)

FLEURS

Gemini 1.5 Flash 8B

86.4%

Gemini 1.5 Flash

9.6%

Gemini 1.5 Pro

6.7%

Current model

Other models

Avg (34.2%)

Video-MME

Gemini 1.5 Pro

78.6%

Gemini 1.5 Flash

76.1%

Gemini 1.5 Flash 8B

66.2%

Current model

Other models

Avg (73.6%)

Vibe-Eval

Gemini Pro 2.5 Experimental

69.4%

56.3%

53.9%

48.9%

40.9%

Current model

Other models

Avg (53.9%)

XSTest

Gemini 1.5 Pro

98.8%

Gemini 1.5 Flash

97.0%

Gemini 1.5 Flash 8B

92.6%

Current model

Other models

Avg (96.1%)

Providers Pricing Coming Soon

We're working on gathering comprehensive pricing data from all major providers for Gemini 1.5 Pro. Compare costs across platforms to find the best pricing for your use case.

OpenAI

Anthropic

Google

Mistral AI

Cohere

Share your feedback

Hi, I'm Charlie Palars, the founder of Deepranking.ai. I'm always looking for ways to improve the site and make it more useful for you. You can write me through this form or directly through X at @palarsio.

Your feedback helps us improve our service

Gemini 1.5 Pro

Model Specifications

Core Specifications

Capabilities & License

Resources

Performance Insights

Detailed Benchmarks

Math

GSM8K

Coding

HumanEval

Reasoning

DROP

Hellaswag

Knowledge

MMLU

MATH

GPQA

Non categorized

MMLU-Pro

PhysicsFinals

AMC_2022_23

MGSM

Natural2Code

HiddenMath

MRCR

WMT23

MMMU

MathVista

FLEURS

Video-MME

Vibe-Eval

XSTest

Providers Pricing Coming Soon

Share your feedback

Stay Ahead with AI Updates