Important papers since 2022: methods and cultural questions

DALL·E 2 combines CLIP representations with a diffusion-based image prior and decoder, improving realism and correspondence with text. Its research preview, beta and later access without a waitlist expand during 2022. The master emphasises this product transition as a step in the international popularization of text-to-image generation.

The original LoRA paper appears in 2021, reducing fine-tuning costs by freezing a main model and training low-rank update matrices. During 2022–2023, the Stable Diffusion community adapts this into small exchangeable modules for styles, characters, clothing, poses and likenesses. Style and identity can circulate as lightweight model files as well as names in prompts.

Imagen uses a large text encoder and diffusion decoders. The research highlights gains in image–text alignment from scaling the language model. It develops a direction that later multimodal systems also pursue: improvements in understanding language can change the visual capabilities and controllability of image generation.

Parti explores an autoregressive approach alongside diffusion, generating image tokens as another kind of language. It shows that the expansion of text-to-image generation in 2022 did not follow a single technical route. Token-based modelling remains a parallel way to organize the relation between descriptions and visual outputs.

EDM separates sampling, training and network preconditioning into comparable design choices, making speed and image quality questions of method rather than model branding.

DPM-Solver designs a dedicated solver for diffusion ordinary differential equations, reducing network evaluations and adding a history of sampling methods behind model releases.

Rectified Flow formulates generation and distribution transfer through increasingly straight transport paths, supplying a theoretical source for later flow-based image models independently of product announcements.

Flow Matching trains continuous flow models by regressing vector fields along conditional probability paths, bringing diffusion and alternative transport paths into a shared methodological framework.

DiT replaces the usual U-Net backbone with a transformer operating on latent patches and studies compute scaling, providing a key architectural comparison for later generative models.

Muse learns masked prediction over discrete image tokens with parallel decoding, documenting a text-to-image route distinct from common diffusion sampling and token-by-token autoregression.

Consistency Models investigates one- and few-step generation through either distillation or standalone training, supplying a methodological background for later interactive image-generation workflows.

SDXL develops composition, detail, style and resolution while retaining a model ecosystem that can be fine-tuned and combined with other tools. The master positions it within the Stable Diffusion community's movement from rapid experimentation toward more established production practices.

PixArt-α focuses on diffusion Transformers, pretrained text encoders and efficient training. It is an open research node leading into the wider development of Transformer- and flow-based image systems in 2024. Its significance concerns both architectural choices and the resources needed to train visual models.

Latent Consistency Models use consistency distillation in latent space to generate high-resolution images in a small number of steps, including two to four in the reported approach. Speed changes the possible interface: generation closer to real time can participate more readily in drawing, design, live presentation and interactive software.

Stable Diffusion 3 combines a Transformer-based architecture with flow matching and emphasises multi-subject prompts, text and image quality. It represents the further incorporation of Transformer approaches into mainstream image generation. The associated research paper provides a technical account distinct from the product announcement.

VAR reformulates visual autoregression as coarse-to-fine next-scale prediction, providing a distinct image-generation route to compare with diffusion transformers.

The paper models continuous-token distributions with diffusion, connecting autoregression to continuous latents and showing that autoregressive image generation need not require discrete quantisation.

REPA aligns intermediate denoising representations with pretrained visual encoders, investigating how visual representations can support diffusion and flow-transformer training.

SANA combines deep latent compression, linear attention and efficient sampling for high-resolution synthesis, connecting model design with computation and local production conditions.

Janus-Pro studies training, data and model scaling in a unified multimodal system, providing follow-up evidence for the Janus approach to understanding and image generation.

MeanFlow characterises generative flows through interval-average rather than instantaneous velocity and trains one-step models from scratch, extending the history of few-step generation.

Textual Inversion learns a new text embedding without retraining an entire model. Visual concepts, objects and styles can be represented through a new composable word within the system. This is an effective method of personalisation, not a claim that the nature of artistic style can be reduced to a token.

Prompt-to-Prompt studies cross-attention between words and image layout, using prompt changes for local replacement and broader transformation. It connects image synthesis with the problem of editable continuity.

DreamBooth fine-tunes a pretrained text-to-image model using a small set of subject images, associating a unique identifier with that subject. It brings preservation of identity into the generative process and supplies technical context for later customization of people, characters and products, as well as disputes concerning likeness.

InstructPix2Pix uses GPT-3 and Stable Diffusion to synthesize training examples, then trains a diffusion model conditioned on editing instructions. It shifts the interaction from making a picture in a single generation toward changing an existing image through language, connecting generative modelling with iterative visual editing.

Null-text Inversion optimises unconditional text embeddings to invert real images, connecting prompt-based editing to existing photographs without retraining the model weights.

GLIGEN adds gated layers to inject grounding conditions into a pretrained diffusion model, using text and bounding boxes to control objects and layout beyond a prompt alone.

ControlNet freezes a pretrained diffusion backbone and learns spatial conditions through an additional network. It extends control from textual semantics toward geometry and composition. This is one technical condition for more specialized production workflows, in which artists or designers need to specify structure as well as subject matter.

Alongside ControlNet, T2I-Adapter exemplifies a move toward fine-grained external control through modular attachments without fully retraining a foundation model. Image production becomes a combination of a base model and adapters. The node situates controllability in a growing ecosystem of interoperable components.

DragGAN uses a GAN's feature space for point-based manipulation, demonstrating direct interaction with a generative image manifold. Although diffusion becomes increasingly prominent, this approach remains relevant to the development of editing interfaces in which users move elements visually instead of specifying every change through text.

IP-Adapter uses decoupled cross-attention to add lightweight image-prompt capabilities to pretrained diffusion models. Reference-image control can coexist with text prompts and ControlNet, supporting workflows for portraits, fashion and identity that combine several types of guidance rather than relying on a description alone.

InstantID combines facial embeddings with structural conditions to support efficient identity preservation in diffusion models. It marks a move from training specifically on a person, as in DreamBooth-style workflows, toward generation guided by references. The technique is directly relevant to later questions of likeness and consent.

DreamFusion uses a two-dimensional diffusion prior to optimise a three-dimensional representation, extending text-driven synthesis toward objects that can be viewed and lit from different directions.

Zero-1-to-3 conditions novel views on a single image and a relative camera change, connecting image priors with 3D reconstruction while exposing the problem of inferred unseen surfaces.

ProlificDreamer introduces variational score distillation to address oversaturation, oversmoothing and limited diversity, documenting a methodological change in text-to-3D generation.

3D Gaussian Splatting represents scenes with optimisable 3D Gaussians and renders new views efficiently. It is spatial-image infrastructure, not itself a text-to-3D generator.

TRELLIS uses structured latents to connect text or image conditions with meshes, radiance fields and Gaussians, examining representation and editability in 3D asset generation.

Video Diffusion Models extends image denoising into time, combining image and video training with clip extension. It supplies a research account of temporal continuity behind later video-generation services.

AnimateDiff introduces motion modelling as a pluggable component within personalized text-to-image diffusion systems. Existing character models, styles and community assets can participate in animated outputs. This extends generation from still images toward video while retaining parts of the earlier image-model ecosystem.

Stable Video Diffusion makes latent video diffusion available to researchers and developers, extending an open generative ecosystem from still images toward video. It also points toward increasingly shared infrastructure between image and video models, including software for arranging and adapting generation processes.

Lumiere uses a space–time U-Net that processes the full video duration in a model pass, offering an alternative to sparse keyframes followed by temporal interpolation.

FramePack packs past-frame context by importance and studies generation drift, making memory and computation in longer video generation an independent research entry.

Self Forcing trains video models on their own generated context, addressing the mismatch between ground-truth training frames and self-generated inference history in streaming generation.

AudioLDM applies latent diffusion to general sound synthesis through language–audio representations, extending the research timeline to environmental audio and sound editing.

MusicGen organises multiple streams of compressed music tokens through a single language model, using text or melody conditions and adding sonic representation and control to the timeline.

GenEval decomposes text–image alignment into object co-occurrence, position, count and colour, making “good generation” more specific than a single overall quality score.

VBench separates video quality into 16 dimensions, including subject consistency, motion, flicker and spatial relations, and checks metrics against human annotations.

LAION-5B makes a large image–text pair index and filtering methods available, providing data infrastructure relevant to Stable Diffusion and related ecosystems. It also becomes central to disputes about training sources, artists' names and copyright. The master reads this as a shift in which the dataset itself becomes a political object; an index does not confer ownership of linked images.

This paper tests memorisation through a generate-and-filter procedure, making training-image reproduction an empirical research question rather than an inference from visual resemblance alone.

Glaze studies small image perturbations intended to disrupt style-mimicking fine-tuning and includes artist studies, making refusal of model learning part of the technical history of generative art.

Epstein and colleagues outline a research agenda spanning aesthetics and culture, ownership and credit, creative labour and media ecosystems, extending discussion beyond model performance.

Nightshade studies prompt-specific training-data poisoning and discusses creator resistance to scraping that ignores opt-out requests, connecting technical research with disputes over data control.