- Learn Prompting's Newsletter
- Posts
- New Gemini 2.5 Pro: The Undisputed LMSYS Champion
New Gemini 2.5 Pro: The Undisputed LMSYS Champion
Google's newest Gemini model sweeps every category, becoming the first model to ever do so
Learn Prompting Newsletter
Your Weekly Guide to Generative AI Development
Gemini 2.5 Pro: Google's New AI Model Dominates LMSYS Leaderboards
Learn how Gemini made history with its performance across text, vision, and WebDev
Hey everyone!
Google has just released a new Gemini 2.5 Pro model that has topped the LMSYS leaderboards. The new Gemini 2.5 Pro (preview 0506) model is now dominating the competitive leaderboard by being ranked number one in five of the seven evaluation categories.
What does this mean for you? Expect significantly improved code generation, more intuitive reasoning, and a model that can handle a variety of tasks. In their blog post, Google explains that the model “achieves leading performance on our junior-dev evals”.
How to Access Gemini 2.5 Pro
Currently this new model is available in three places: the Gemini app, Google AI Studio and the Gemini API.
Gemini App
For non-developers, I’d recommend heading to the Gemini app to try this new model. You can find 2.5 Pro by heading to the top left corner and opening the drop down menu. Here you can find all available models and select “2.5 Pro (preview)”.
Google AI Studio
For developers looking to experiment with the new 2.5 Pro model, I’d suggest using Google AI Studio. AI Studio is an entirely free platform where you can work with all Gemini models.
To get started:
Head to AI Studio and sign in with your Google account.
Select “Run Settings” in the top right corner and click on the model name. In the example below, my default is “Gemini 2.0 Flash”.
Select “Gemini 2.5 Pro Preview 05-06” from the dropdown.
Gemini API
If you’re looking to build with this model, the Gemini API has already been updated to work with the latest model. For any developers already working with the 2.5 Pro model, no changes are necessary on your end and the price will remain the same.
“For developers already using Gemini 2.5 Pro .. the previous iteration (03-25) now points to the most recent version (05-06), so no action is required to use the improved model, and it continues to be available at the same price” - (Gemini 2.5 Pro blog announcement).
Comparison to Other Models
The new 2.5 Pro preview 0506 model performs incredibly well when compared to the past version of Gemini 2.5 Pro as well as other state-of-the-art LLMs. We can see how they compare by looking at the LMSYS leaderboards. For those who aren’t familiar, LMSYS is a respected third-party evaluation platform which allows users to compare and rate model responses through blind testing.
Gemini 2.5 Pro is currently #1 on the overall LMSYS leaderboard.
Currently Gemini 2.5 Pro ranks number one in five separate leaderboards. These include Text (Language), Overview, Web Development, Vision and Search. While ranking first in a single category is impressive, being the top model in five different categories shows how powerful this model really is. This success was significant enough for the LMSYS team to announce on Twitter that “This is the first-ever sweep across text, vision, and WebDev by any model”.
🚨Breaking: @GoogleDeepMind’s latest Gemini-2.5-Pro is now ranked #1 across all LMArena leaderboards 🏆
Highlights:
- #1 in all text arenas (Coding, Style Control, Creative Writing, etc)
- #1 on the Vision leaderboard with a ~70 pts lead!
- #1 on WebDev Arena, surpassing Claude— lmarena.ai (formerly lmsys.org) (@lmarena_ai)
3:22 PM • May 6, 2025
Gemini 2.5 Pro in Action
Of course, the real test for these models lies in what people can create with them. Developers have been shipping some incredible projects using the latest 2.5 Pro model. From immersive 3D environments with complex physics to fun 2D platformer games, vibe coding with Gemini has seen some exceptional products. To find an extensive list of projects created with Gemini, check out this Twitter thread from Google AI Developers. They’ve brought together some of the most complex simulations, games, websites, and visualizations created with Gemini 2.5 Pro.
Vibe coded cubes inside cubes
— Google AI Developers (@googleaidevs)
9:00 PM • May 7, 2025
Reply