Sparsity
Last Updated: July 29, 2026 | By Mihail Sebastian | AI Dictionary
The condition where most values in a dataset, matrix, or model's weights are zero, which lets systems store and compute only the entries that matter.
What is Sparsity?
Sparsity is the condition in which most values in a dataset, matrix, or model are zero or absent.
Rather than a defect, sparsity is an opportunity: a system that stores and computes only the non-zero entries skips most of the work. Sparse formats such as Compressed Sparse Row keep just the values that exist and where they sit.
Types of Sparsity
- Data sparsity arises when high-dimensional data has few active entries per row. A document contains a tiny fraction of the vocabulary, so its word-count vector is almost all zeros.
- Model sparsity is engineered: pruning sets a network’s smallest weights to exactly zero, shrinking the model. Alongside quantization, it is one of the two standard ways to compress a network for cheap inference.
Example of Sparsity
A streaming service holds a ratings matrix: one row per user, one column per film, 50,000 films wide. Most users have rated only a few dozen titles, so well over 99% of the matrix is empty.
Stored densely, the matrix would need one cell per user-film pair, almost all of them zeros. Stored sparsely, the service keeps only the ratings that exist (user, film, score) and cuts storage by orders of magnitude.
The recommendation algorithm exploits the same structure. Matrix factorization iterates over the observed ratings alone, so training time scales with the number of ratings rather than the size of the full matrix.
Related AI terms: Quantization · Matrix Factorization · Dimensionality Reduction · Inference
Did you like the Sparsity gist?
Learn about 250+ need-to-know artificial intelligence terms in the AI Dictionary.
Mihail Sebastian — Writes about AI governance, regulation, and the technology behind them. Placeholder bio — replace with a real credential line. About