Papers Explained 186: Grok
Grok is a 314B Mixture-of-Experts model, with 25% of the weights active on a given token, modeled after the Hitchhiker’s Guide to the Galaxy, hence designed to answer questions with a bit of wit and has a rebellious streak. It will also answer spicy questions that are rejected by most other AI systems . It has real-time knowledge of the world via the 𝕏 platform.
Grok-1 displayed strong results, surpassing all other models in its compute class, including ChatGPT-3.5.
Grok 1.5
Grok-1.5 is an advancement over grok, capable of long context understanding up to 128k tokens and advanced reasoning.
Grok-1.5 can handle longer and more complex prompts, while still maintaining its instruction-following capability, In the Needle In A Haystack (NIAH) evaluation, Grok-1.5 achieved powerful perfect retrieval results for embedded text within contexts of up to 128K tokens.
Grok 1.5 V
Grok-1.5V, is the first multimodal model in the grok series. In addition to its strong text capabilities, Grok 1.5V can process a wide variety of visual information, including documents, diagrams, charts, screenshots, and photographs.
Grok 2 and Grok 2 Mini
Grok-2 is a frontier language model with state-of-the-art capabilities in chat, coding, and reasoning on par with Claude 3.5 Sonnet and GPT-4-Turbo. Grok-2 mini is a small but capable sibling of Grok-2.
On the lmsys arena Grok-2 outperforms both Claude 3.5 Sonnet and GPT-4-Turbo.
Both Grok-2 and Grok-2 mini demonstrate significant improvements over the previous Grok-1.5 model.
- * GPT-4-Turbo and GPT-4o scores are from the May 2024 release.
- † Claude 3 Opus and Claude 3.5 Sonnet scores are from the June 2024 release.
- ‡ Grok-2 MMLU, MMLU-Pro, MMMU and MathVista were evaluated using 0-shot CoT.
- § For MATH, maj@1 results are presented.
- ¶ For HumanEval, pass@1 benchmark scores are reported.
Hungry for more insights?
Don’t miss out on exploring other fascinating threads in this series. Simply click here and uncover the state-of-the-art research!
Do Subscribe for weekly updates!!
