Llama 4: What You Need to Know

Meta’s latest open-source, multimodal model, Llama 4, was announced in April. Known as the Llama 4 Herd, the family of open large language models (LLMs) includes:

• Llama 4 Scout: Optimized for inference and long-context processing.
• Llama 4 Maverick: Ideal for general use scenarios, complex reasoning and creative tasks.
• Llama 4 Behemoth: Designed for distillation.
• Llama 4 Reasoning: Engineered for reasoning models.

The Llama 4 family marks a new chapter in open-weight foundation models, with versions optimized for reasoning, long context processing and efficient distillation. Prior Llama models remain available and widely used, but Llama 4 sets a higher bar for compute efficiency and multimodal performance.

What’s New in Llama 4?

Here are three standout features that define Llama 4, and why they matter for teams deploying real-world AI systems at scale.

Mixture-of-Experts (MoE) Architecture

Llama 4 introduces Meta’s first production-grade MoE (Mixture-of-Experts) architecture. Unlike dense models that activate all parameters for each token, MoE selectively routes tokens through a small subset of expert sub-networks, reducing per-token compute while enabling massive model scale.

Llama 4 Scout uses 17B active parameters across 16 experts and fits on a single NVIDIA H100 GPU using quantized inference. Llama 4 Maverick retains the same active size but spreads computation across 128 experts, totaling over 400B parameters and requiring all experts to remain in memory for routing. MoE layers are interleaved with dense layers to preserve generalization while improving training efficiency.

While these models offer high performance at lower serving cost per query, they impose new infrastructure demands. Token routing, expert loading and cross-layer communication all benefit from GPU clusters with high-bandwidth memory and ultra-low latency interconnects. Llama 4 Maverick is typically deployed on multi-GPU hosts or across distributed systems with NVIDIA Quantum-2 400Gb/s InfiniBand networking to ensure inference efficiency and scale.

MoE models represent a step-function shift in deployment patterns, requiring not just powerful GPUs, but thoughtful engineering of memory, bandwidth and topology.

Native Multimodal Support

Another major step forward is native multimodality. Llama 4 supports multimodal inputs, text, image and video, through early fusion. Unlike retrofitted models, it integrates all input types directly into the same token stream and model backbone. This design improves contextual understanding but requires significantly more GPU memory and bandwidth during both training and inference.

Multimodal inputs increase token count and model depth per query, making high-throughput inference impractical without high-bandwidth memory and fast interconnects — especially at large context lengths.

Extended Context Lengths

Llama 4 introduces massive context capacity — up to 10 million tokens in Llama 4 Scout and 1 million in Llama 4 Maverick. This enables advanced use cases like multi-document reasoning, long-form code analysis and cross-modal memory over large timelines.

Such extreme context lengths dramatically increase the active memory footprint and attention bandwidth per query. For real-time performance, these models require GPUs with high-bandwidth memory and low-latency interconnects like NVIDIA InfiniBand to manage sequence state across devices.

What You Need to Run It

Running Llama 4 at scale requires more than just powerful GPUs, it demands thoughtful infrastructure design across compute, memory and networking. Below, we outline the infrastructure requirements for Llama 4 Scout and Maverick, two key models in the Llama 4 Herd, and what it takes to run them effectively in production environments.

Llama 4 Scout

Llama 4 Scout can run on a single NVIDIA H100 for basic use with quantization, but production deployments, especially with long context or multimodal workloads, require 4–8x H100 or 2–4x H200 configurations. Extended context (up to 10M tokens) and image inputs significantly increase memory and bandwidth requirements.

Llama 4 Maverick

While operational on a single NVIDIA H100 host (8x H100 GPUs) for dev workloads, Llama 4 Maverick’s 400B parameter footprint and 128-expert MoE architecture benefit from 8x H200 or multi-node H100 or H200 deployments. Optimal inference requires high-bandwidth interconnects like NVIDIA NVLink to coordinate expert routing across devices.

Why IREN Cloud™ is Ready
Llama 4 was designed for infrastructure with serious scale, and IREN Cloud™ delivers. IREN operates clusters of NVIDIA H100, H200 and Blackwell GPUs. These clusters are networked using ultra-fast NVIDIA Quantum-2 400Gb/s InfiniBand and NVLink interconnect technologies, delivering up to 3.2 TB/s of throughput to support high-bandwidth, low-latency AI workloads.

IREN’s AI data centers are engineered for power-dense compute: high-bandwidth networking, precision power and cooling, and ultra-fast interconnects, ideal for long-context inference, multimodal pipelines, and distributed MoE models.

You can deploy Llama 4 today using:

1. IREN Cloud’s Reserved Instances: Dedicated H100, H200, or Blackwell nodes
2. IREN Colocation Data Center Services: Host your own hardware in GPU-optimized environments
3. IREN Build-To-Suit Data Centers: Design and build your custom AI-ready facility using IREN’s 2.75GW of available capacity across our Texas campuses.

Whether you're running exploratory workloads or scaling to 100x users, IREN’s AI data centers gives you the power and control to deploy Llama 4 at production scale.