Intermediate · NLP

Long Context

Visual diagram · (in preparation) · Math · (in preparation) · Worked example · 3 difficulty levels.

TL;DR. The ability of an LLM to process and maintain coherence over very long input sequences or conversations, beyond typical token limits.

Technical Definition

The ability of an LLM to process and maintain coherence over very long input sequences or conversations, beyond typical token limits.

How it works

Achieving long context is a significant challenge and advancement in LLM development. It allows models to maintain a deep understanding across extended dialogues, entire documents, or even multiple articles. This capability is crucial for applications like comprehensive summarization, detailed question answering, and maintaining consistent persona across prolonged interactions, overcoming the limitations of smaller context windows.

Related Concepts

• Context Window — The maximum number of tokens a model can process in a single input, determining information capacity.
• Retrieval augmented generation (RAG) — LLMs that can access and use external information to improve their responses.
• Transformer Architecture — A neural network architecture, predominantly used in NLP, that relies heavily on self-attention mechanisms to process sequential data.