Alexander Lewis 
Feb 16, 2024
Only days after rebranding its generative artificial intelligence (GenAI) tool as Gemini, originally launched as Google Bard, the search engine giant Google has announced the tool's revamped version: Gemini 1.5.
Initially releasing for select testers, Google says the upcoming model of next-generation AI chatbot would ensure “dramatically enhanced performance.”
Meanwhile, the most significantly crafted upgrade in the latest version is a greatly expanded context window.
Read more: Text to video generator — OpenAI working on new AI model Sora
The Gemini 1.5 Pro, a middle-tier variant of Gemini family, has a standard context window that accommodates 128,000 tokens — same as we have in GPT-4 Turbo — while the base variant, Gemini 1.0, offered a standard window to fit 32,000 tokens.
The augmented capacity of Gemini's context window can now contain over 700,000 words, codebases with over 30,000 lines of code, 11 hours of audio, or 1 hour of video. On the other hand, Claude 2.1 offers 200,000 tokens, surpassing both Gemini and GPT-4 Turbo.
However, the most outstanding fact is that the company has run up to 1 million tokens in the production phase, while its working to offer the model to some early testers, while it has “successfully tested up to 10 million tokens” that were text-based.
These advancements have been enabled by a new Mixture-of-Experts (MoE) architecture, in which models are divided into smaller "expert" neural networks. This results in Gemini 1.5 being more efficient for both training and serving.
In terms of performance, 1.5 Pro outperforms 1.0 Pro on 87% of the benchmarks in text, code, image, audio, and video evaluations. It even demonstrates a similar level of performance to 1.0 Ultra.
Gemini 1.5 Pro (with a 128,000 token context window) is being launched as a limited preview for developers and enterprise customers through AI Studio and Vertex AI. It is described as experimental during this period.
