Mathematics and probability · MLD-0244

Cross-entropy

Expected negative log probability under a predicted distribution, averaged over a target distribution.

Original reference page · detailed expansion pending. Original definitions, examples, notes and citations are preserved below. New examples are illustrative, not benchmark results.

Original example

Train a classifier against one-hot target labels.

Original caveat

Often a loss; its units depend on the logarithm base.

Context and aliases

Level: Foundational. Stages: Theory · Training · Evaluation.

Tags: ML · DL · SLM · LLM.

Aliases: None recorded.

Related concepts

Entropy
Kullback-Leibler divergence

Concepts linking here

Binary cross-entropy
Bits per character
Categorical cross-entropy
Entropy
Focal loss
Kullback-Leibler divergence
Label smoothing
Log loss
Logistic regression
Loss function
Negative log-likelihood
Perplexity

Original references

Deep Learning: online textbook ↗ · Consulted
Mathematics for Machine Learning ↗ · Reference link
Original reference URL ↗

Open original record, export or annotate →