CUDA (Compute Unified Device Architecture)
Last Updated: July 29, 2026 | By Mihail Sebastian | AI Dictionary
NVIDIA's parallel computing platform and programming model that lets ordinary code run across thousands of GPU cores – the layer beneath most deep learning.
What is CUDA (Compute Unified Device Architecture)?
CUDA is a parallel computing platform and programming model from NVIDIA that lets developers run general-purpose code on the company’s graphics processing units (GPUs).
A GPU contains thousands of small cores built to apply the same operation to many pieces of data at once. CUDA exposes that hardware to programmers, and because neural network training is mostly matrix arithmetic of exactly that shape, CUDA became the software foundation of modern deep learning.
How CUDA Works
A CUDA program divides work between the CPU and the GPU. The CPU handles ordinary sequential logic; the performance-critical pieces are written as kernels, functions that thousands of GPU threads execute in parallel. Data moves into GPU memory, the kernels run, and results copy back.
Few machine learning practitioners write kernels themselves. Frameworks such as PyTorch and TensorFlow call NVIDIA’s CUDA libraries (cuDNN for neural network operations, cuBLAS for linear algebra) behind the scenes, so moving a training job from CPU to GPU is a one-line change.
The catch is that CUDA runs only on NVIDIA hardware. Because so much AI software is built on it, that dependence shapes who buys which chips – a lock-in worth understanding when a project or procurement decision assumes the GPU market is interchangeable.
Example of CUDA
A team trains an image classifier. Each training step multiplies large matrices millions of times, and every multiplication is independent of the others, so the work parallelizes perfectly.
On a CPU with a few dozen cores, one training run takes weeks. With CUDA, PyTorch loads each data batch into GPU memory, launches kernels that compute the forward pass across thousands of threads at once, runs the backward pass the same way, and updates the weights without the data ever leaving the GPU.
The same run finishes in days or hours. Nothing about the model changed; the arithmetic simply moved to hardware built for it.
Related AI terms: GPU Memory · TPU · Deep Learning · PyTorch · TensorFlow
Did you like the CUDA (Compute Unified Device Architecture) gist?
Learn about 250+ need-to-know artificial intelligence terms in the AI Dictionary.
Mihail Sebastian — Writes about AI governance, regulation, and the technology behind them. Placeholder bio — replace with a real credential line. About