Sequence architectures and attention · MLD-0365
Encoder-only model
A model centered on an encoder producing contextual input representations.
Original reference page · detailed expansion pending. Original definitions, examples, notes and citations are preserved below. New examples are illustrative, not benchmark results.
Original example
Build embeddings or classification predictions.
Original caveat
It is not usually a standard left-to-right generator.
Context and aliases
Level: Intermediate. Stages: Training · Inference · Theory.
Tags: DL · SLM · LLM · NLP.
Aliases: None recorded.
Related concepts
Concepts linking here
Original references
- Attention Is All You Need ↗ · Consulted
- Transformers glossary ↗ · Consulted
- PyTorch neural-network layers ↗ · Reference link
- Original reference URL ↗