Sequence architectures and attention · MLD-1109
Sequence length
The number of sequence elements processed in a model input or computation.
Original reference page · detailed expansion pending. Original definitions, examples, notes and citations are preserved below. New examples are illustrative, not benchmark results.
Original example
Train on sequences of 2,048 tokens.
Original caveat
It may include padding or exclude it depending on reporting.
Context and aliases
Level: Intermediate. Stages: Training · Inference · Theory.
Tags: DL · SLM · LLM · NLP.
Aliases: None recorded.
Related concepts
Concepts linking here
Original references
- Attention Is All You Need ↗ · Consulted
- Transformers glossary ↗ · Consulted
- PyTorch neural-network layers ↗ · Reference link
- Original reference URL ↗