Tokenization and text representation · MLD-1111
Sequence padding
Adding placeholder positions so variable-length sequences can share tensor shapes.
Original reference page · detailed expansion pending. Original definitions, examples, notes and citations are preserved below. New examples are illustrative, not benchmark results.
Original example
Batch texts of unequal lengths.
Original caveat
Padding positions often need masking in both attention and loss.
Context and aliases
Level: Intermediate. Stages: Data · Training · Inference.
Tags: NLP · SLM · LLM.
Aliases: PAD token · sequence padding · Padding.
Related concepts
Concepts linking here
Original references
- Hugging Face: chat templates ↗ · Reference link
- Transformers glossary ↗ · Consulted
- Hugging Face: tokenization algorithms ↗ · Consulted
- Original reference URL ↗