Language-model training and adaptation · MLD-0681

LoRA rank

The dimension of the low-rank factors used in a LoRA update.

Original reference page · detailed expansion pending. Original definitions, examples, notes and citations are preserved below. New examples are illustrative, not benchmark results.

Original example

Choose how much capacity a LoRA adapter has.

Original caveat

Higher rank increases trainable size but does not guarantee better results.

Context and aliases

Level: Intermediate. Stages: Training · Adaptation · Evaluation.

Tags: DL · SLM · LLM · NLP.

Aliases: Adapter rank · r (LoRA).

Related concepts

Concepts linking here

Original references

Open original record, export or annotate →