How AI Learned to Paint: Style Transfer to Diffusion
Last Updated: September 10, 2026 | By Mihail Sebastian | AI Experiments
The decade AI learned to paint: from 2015 style transfer to diffusion models, and why the training data behind the images is now a legal fight.
In 2015, researchers at the University of Tübingen showed that a neural network could repaint a photograph in the brushwork of Van Gogh’s Starry Night. The images spread across the internet before most people who shared them had ever heard the term “neural network.”
Ten years later, machines turn a sentence into a finished image in seconds, artists are suing the companies that built them, and regulators are writing labeling rules for the output. The story of how AI learned to paint is short, well documented, and ends somewhere nobody at the start expected: in court.
The Parlor Trick That Started It
The 2015 technique was called neural style transfer. Leon Gatys and his colleagues noticed that a network trained to recognize objects represents an image’s content and its texture separately, at different layers. Separate the two and you can recombine them: the structure of your photo, the color and stroke statistics of a famous painting.
The original method was slow. It optimized each output image from scratch, which took minutes per photo on a good GPU, and follow-up work soon trained feed-forward networks that applied a given style in a single pass. That speedup is what made the technique a consumer product.
A year later the trick reached phones. Prisma, launched in June 2016, applied the effect to selfies and street shots, and millions of people downloaded it within weeks. For one summer, everyone’s vacation photos looked like Expressionist woodcuts.
It is worth being honest about what this was. Style transfer created nothing: it needed two existing images and produced a mathematical blend of them. The network had not learned what Van Gogh saw, only which textures sit where, so the result was rearrangement, not painting.
GANs Learn to Invent
The next step dropped the second image entirely. Generative adversarial networks, introduced in 2014, train a generator against a discriminator until the generator’s fakes pass for real. By the decade’s end, GAN demo sites were serving a photorealistic face on every refresh, each belonging to a person who had never existed.
The art world took notice in October 2018, when Christie’s in New York auctioned Edmond de Belamy, a smeared portrait produced by a GAN that the Paris collective Obvious had trained on historical paintings. Against a pre-sale estimate under $10,000, it sold for $432,500. The signature in the corner was a fragment of the model’s loss function.
Technically, something real had changed. A GAN does not blend two inputs; it learns the distribution of its training set and samples new instances from it. That is invention, though of a narrow kind: a model trained on faces makes faces, and nothing else.
Diffusion Changes Everything
Diffusion models broke the narrowness. They start from pure noise and remove it step by step, steered by a text description, until an image emerges. Connect that process to a language model’s understanding of text and any style you can name becomes a style you can request.
Three systems arrived within months of each other in 2022: OpenAI’s DALL-E 2 in April, Midjourney’s open beta in July, and Stable Diffusion in August, released with open weights anyone could download. Generative AI for images went from research demo to consumer product in a single year, and image generation has since become a native feature inside general-purpose assistants rather than a separate tool.
What fed these systems was billions of image-text pairs scraped from the public web, working artists’ portfolios included. That detail produced the quality of the results. It also produced the fight that followed.
The Fight Over the Training Data
For artists the problem was concrete: their own names worked as prompts. A living illustrator could watch a model generate endless images “in the style of” them, from a system trained on work copied without permission or payment.
The disputes reached court quickly. Getty Images sued Stability AI in 2023, in both the UK and the US, alleging its licensed photographs were copied to train Stable Diffusion, and groups of artists filed suits of their own the same year. The cases turn on whether model training on copyrighted works is fair use or infringement, and no settled answer has emerged.
Artists also armed themselves technically. Research tools such as Glaze and Nightshade let an artist alter published images so that a model scraping them either cannot extract the style or learns a corrupted version of it: a cloak on the work, or poison in the training pipeline.
Between litigation and sabotage, a middle ground is forming. Some providers honor opt-out signals that exclude flagged works from training, and licensing deals pay rights holders for training access. Neither is universal, but together they sketch what a negotiated settlement between artists and model builders could look like.
Did AI Learn to Paint?
The legal ground under all of this is stranger than the headlines suggest. Style itself is not copyrightable; copyright protects specific works, not a manner of painting. The mimicry that angers artists most is the part the law protects least, while the contested act sits upstream, in the copies made during training.
Ownership of the output is equally unsettled from the other direction. The US Copyright Office has taken the position that purely AI-generated images lack the human authorship copyright requires, so a picture can be too derivative for artists’ comfort and still be owned by no one.
Disclosure obligations are arriving regardless of how the copyright fights resolve. The EU AI Act requires AI-generated and manipulated media to be disclosed as such, so the question “is this image synthetic?” is becoming one the image must answer itself.
So did AI learn to paint? It learned to render. Style transfer borrowed, GANs invented within narrow limits, and diffusion renders anything describable in anyone’s manner, on demand.
Whether rendering counts as painting used to be a question for critics. It is now a question for courts and regulators, because the answer decides who gets paid, who gets credited, and what the machine was allowed to look at while it learned.
