A Developer’s Guide to Attention in vLLM

PyTorch Conference North America 2026

Oct 20, 2026 · 11:45–12:10 · Timezone not confirmed

Attention is no longer just “standard attention.” As models push toward longer context windows, attention has become one of the most active areas of architecture design. New schemes, including sliding windows, sparsity, compression, and linear variants, save KV-cache memory at long contexts, and nearly all modern state-of-the-art models now use hybrid attention, mixing patterns across layers to balance quality with memory use. This talk will give developers a practical guide to how vLLM represents, serves, and optimizes this growing diversity of attention schemes. We will start by reviewing the schemes used in recent models, and then walk through the core vLLM abstractions that make them possible, including attention backends, KV-cache connectors, and the hybrid memory allocator. Together, these allow vLLM to support a wide range of attention implementations while maintaining a unified serving path, efficient batching, and high-performance memory management. Finally, we will cover the recent overhaul of vLLM’s attention abstractions: what changed, why the old assumptions no longer hold, and how the new design makes it easier and cleaner to support emerging architectures.

Session status: scheduled

Room: LL20AB

Track: Introduction

Level: Advanced

Official session / agenda

Saving a session is not registration. Check the official agenda for availability, eligibility and any additional fees.

Speakers and their event affiliations

Parent event

In person · San Jose Convention Center, San Jose, California, United States

This describes the parent event. Confirm the session’s attendance requirements with the organizer.

Companies participating in the parent event

These are event-level relationships, not confirmed sponsorship of this session.

Showing 24 of 130 company relationships. View all on the event page.