WACV 2027

Image Deblending with Diffusion Models

Dan Botchan Ohad Fried Yacov Hel-Or

Reichman University, Herzliya, Israel

x̃¹ recovered“Lemons on the shelf in the supermarket”
α̃ recoveredpolynomial map, 6 parameters
x̃² recovered“A tiger facing the camera”

y the blended input, all we observe

One blended image in, two images and the blending map out. Drag to separate. The method is given only y and one text prompt per source; it never sees the sources or the map.

In short

Two images from one prior

Deblending becomes a joint inverse problem. Diffusion posterior sampling recovers both full source images and the alpha map together, from a single pretrained model.

Prompts settle the ambiguity

One blend has many valid splits. A short text prompt per source, used with cross-negative guidance, tells the sampler which content belongs where.

Coarse in latent space, fine in pixels

Stable Diffusion separates the semantics at 512 × 512. A pixel-space ImageNet diffusion model then restores texture and removes VAE artifacts at 256 × 256.

No training, no ground truth

Nothing is fine-tuned. Three candidate decompositions are sampled and PickScore picks one, so the whole pipeline runs without any reference images.

Abstract

Image deblending is the task of decomposing a composite image back into its source images, which is an ill-posed problem, as a single blended image can be decomposed into multiple plausible source layers. While diffusion models are often used to solve inverse problems, they typically address single-output reconstruction problems. In this work, we formulate deblending as a joint inverse problem, employing diffusion posterior sampling to simultaneously reconstruct two source images, demonstrating that a single diffusion prior is expressive enough to disentangle two full images at once rather than just one. Drawing inspiration from human perception, where semantic understanding helps disentangle visual inconsistencies, we propose a training-free framework that leverages text prompts to constrain this solution space. We address the blind deblending task through a two-stage architecture: text-guided latent diffusion performs semantic separation using prompts as high-level cues, followed by a pixel-space refinement to recover fine details. Stochastic sampling produces multiple plausible decompositions that can be ranked without ground truth. Extensive experiments show that our training-free method achieves state-of-the-art results on unseen domains and extends to spatially-varying alpha maps, while remaining competitive with supervised baselines on standard scalar blending benchmarks.

The forward model

Six numbers describe the blend

The observed image is a per-pixel mix of two sources. Instead of a free value per pixel, which could encode edges, the alpha map is a smooth second-order polynomial over normalized coordinates. The sampler estimates these six coefficients along with the two images. Try them here on the teaser sources.

y = α ⊙ x¹ + (1 − α) ⊙ x²
α(ū, v̄) = a + bū + cv̄ + dūv̄ + eū² + fv̄²
α map (white = x¹, clipped to [0, 1])
y, the blended image
Source 1: lemonsx¹ source 1
Source 2: catx² source 2
Method

A two-stage, training-free pipeline

Both stages run the same guided step: denoise each source, re-blend with α, and update both samples from the data-consistency gradient. Stage 1 also updates the α coefficients every k = 5 steps; Stage 2 keeps α fixed and uses no prompts.

Top row: the blended image y and prompts enter Stage 1 in latent space, which runs from N seeds; PickScore selects one candidate pair, and Stage 2 in pixel space refines it with alpha fixed into the final sources and alpha map. Bottom row: the guided step shared by both stages, where each source is denoised, re-blended with the alpha map, and updated from the data-consistency gradient.
  1. 01 · Initialize

    Start from the blend

    Encode y, duplicate it into two branches, and add noise to an intermediate step ts. The layout of the mixture survives while the branches diverge.

  2. 02 · Stage 1

    Separate semantics

    Stable Diffusion v1.4 with cross-negative prompting: each branch uses its own prompt as positive and the other's as negative. Consistency with y is enforced in pixel space, and α is optimized alongside the sources.

  3. 03 · Select

    Pick a candidate

    Three seeds give three decompositions. A permutation-invariant PickScore ranking against the prompts picks one, using no ground truth.

  4. 04 · Stage 2

    Refine in pixels

    Re-noise the selected pair to tr < ts and resample with a 256 × 256 ImageNet DDPM, unconditional and with α fixed, recovering texture and correcting VAE artifacts.

The full pipeline takes about 50.6–71.0 minutes per input on one NVIDIA RTX 3090.

Qualitative results

Compare against every baseline

Pick a domain, a blending scenario and a pair. Press and hold any result to see the ground truth in its place.

Domain
Blending scenario

Ablation · text guidance

Cross-negative prompts keep the sources apart

Standard classifier-free guidance pulls each branch toward its prompt. Using the other source's prompt as the negative also pushes it away from the wrong content. Raise the guidance strength ω to compare the three strategies.

Pair
Guidance strength
ω = 2.0
Blended inputinput y
Ground truth source 1Ground truth source 2
ground truth
Ablation · two stages

What the pixel stage adds

Stage 1 gets the content right but loses detail through the VAE. Drag the divider to compare its output with the Stage 2 refinement of the same sample.

Pair
Blended inputinput y
Quantitative results

Best on unseen domains, competitive on seen ones

Fruits and Animals are the domains the supervised baselines were trained on; Vehicles and Flowers are held out. TF marks training-free methods. Ours and DoubleDIP each draw three candidates and select one with PickScore, without ground truth. The best value per column is underlined.

Blending scenario

Citation

@inproceedings{botchan2027deblending,
  title     = {Image Deblending with Diffusion Models},
  author    = {Botchan, Dan and Fried, Ohad and Hel-Or, Yacov},
  booktitle = {Proceedings of the IEEE/CVF Winter Conference on
               Applications of Computer Vision (WACV)},
  year      = {2027}
}