Interpretability and mechanistic analysis · MLD-1140
Sparse coding
Representing data with only a small number of active coefficients in a chosen or learned dictionary.
Original reference page · detailed expansion pending. Original definitions, examples, notes and citations are preserved below. New examples are illustrative, not benchmark results.
Original example
Encode hidden activations sparsely.
Original caveat
Sparse representations are not automatically semantic.
Context and aliases
Level: Advanced. Stages: Research · Evaluation.
Tags: ML · DL · SLM · LLM · Interpretability · Safety.
Aliases: None recorded.
Related concepts
Concepts linking here
Original references
- Captum model interpretability documentation ↗ · Reference link
- Tracing the thoughts of a large language model ↗ · Consulted
- Towards Monosemanticity: Decomposing Language Models With Dictionary Learning ↗ · Consulted
- SHAP documentation ↗ · Reference link
- Original reference URL ↗