Dimensionality Reduction
Last Updated: July 29, 2026 | By Mihail Sebastian | AI Dictionary
Turning a dataset with many features into one with fewer, keeping the structure that matters so models train faster and high-dimensional data becomes visible.
What is Dimensionality Reduction?
Dimensionality reduction converts a dataset with many features into one with fewer features, keeping as much of the useful structure as possible. The new columns then stand in for the originals during training, visualization, or storage.
The reason to bother is the curse of dimensionality. As features multiply, the space they span grows faster than any realistic sample fills it, so every pair of points drifts toward the same distance from every other and distance-based methods lose their grip. Fewer, denser dimensions restore the contrast.
Types of Dimensionality Reduction
- Feature selection keeps a subset of the original columns and drops the rest. What survives is still interpretable: a kept feature is still “annual income”.
- Linear projection builds new features as weighted mixes of the old ones. Principal component analysis and matrix factorization both work this way, and both are reversible enough to approximate the original data.
- Nonlinear manifold methods unfold curved structure that no straight-line projection captures. t-SNE and UMAP serve mainly to produce 2D plots; an autoencoder learns a compressed code you can feed to a downstream model.
Applications of Dimensionality Reduction
- Visualization: Projecting embeddings or sensor logs to two axes makes clusters, gaps, and outliers something you can look at rather than infer.
- Correlated inputs: When twenty features measure roughly the same thing, regression coefficients swing wildly between retrainings. Collapsing them into a few components stabilizes the fit.
- Compression and speed: Fewer inputs mean smaller models, less memory, and faster inference, which matters when a model runs on a device rather than a server.
Example of Dimensionality Reduction
A bank builds a credit model on 200 features derived from transaction history. Many overlap heavily, so the team runs PCA and keeps the 20 components that carry most of the variance. Training speeds up and the model’s behavior across retrainings settles down.
Then a rejected applicant asks why. Component 7 is a weighted blend of forty original variables, so nothing in the model maps back to a statement a person can act on. The EU AI Act treats creditworthiness assessment of individuals as high-risk, and adverse-decision explanations are hard to produce from components no one can name.
The fix is usually to model on reduced features but keep the original ones for explainability, or to prefer feature selection when a decision has to be justified to the person it affects.
Related AI terms: Principal Component Analysis · t-SNE · UMAP · Feature Selection · Latent Space
Did you like the Dimensionality Reduction gist?
Learn about 250+ need-to-know artificial intelligence terms in the AI Dictionary.
Mihail Sebastian — Writes about AI governance, regulation, and the technology behind them. Placeholder bio — replace with a real credential line. About