Systems and efficient execution · MLD-0436
Float8
A family of eight-bit floating-point formats used in supported low-precision computation.
Original reference page · detailed expansion pending. Original definitions, examples, notes and citations are preserved below. New examples are illustrative, not benchmark results.
Original example
Reduce memory traffic in suitable model operations.
Original caveat
FP8 is not one universal format or a guaranteed accuracy-preserving switch.
Context and aliases
Level: Advanced. Stages: Training · Inference · Deployment.
Tags: DL · LLM · SLM · Systems.
Aliases: FP8.
Related concepts
Original references
- PyTorch Fully Sharded Data Parallel ↗ · Consulted
- PyTorch documentation ↗ · Reference link
- vLLM documentation ↗ · Consulted
- Original reference URL ↗