Foundations · MLD-1131

Small language model

A comparatively compact language model designed for lower resource costs or narrower deployment needs.

Expanded guide · editorial review 2026-11-01. Original definitions, examples, notes and citations are preserved below. New examples are illustrative, not benchmark results.

Prerequisites

How it works

A small language model uses the same broad modeling ideas as larger language models with a smaller parameter or deployment footprint. SLM has no universal size boundary. Data selection, training quality, architecture and task specialization can matter as much as a size label. A smaller model can still produce incorrect or unsafe outputs.

Worked example

Consider an offline assistant that maps a short maintenance report to a fixed set of labels. A compact model may suit the device, but compare it against a simple classifier and a larger reference model on the actual reports. This is a proposed evaluation, not a recommendation for a specific product.

A useful estimate

Weight storage alone is approximately parameter_count multiplied by bits_per_weight divided by eight. Runtime memory also includes activations, caches, metadata and framework overhead.

What to test

Measure accuracy, peak memory, first-response latency and sustained throughput on the intended hardware. Include longer prompts, uncommon terminology and unsupported requests.