【中英双字】一篇文章教你怎么选择Stable Diffusion扩散采样器的类型以及各个扩散器的功能详解教学,再也不用担心不知道怎么选



As we saw in the article How Stable Diffusion works, when we ask Stable Diffusion to generate an image the first thing it does is generate an image with noise and then the sampling process removes noise through a series of steps that we have specified. It would be something like starting with a block of white marble and hammering it for several days until you get Michelangelo's David.

Several algorithms come into play in this process. The one known as sampler is in charge of obtaining a sample from the model that we are using in Stable Diffusion on which the noise estimated by the noise predictor is applied. It then subtracts this sample from the image it is cleaning, polishing the marble in each step.

This algorithm handles the how, while the algorithm known as the noise scheduler handles the how much.

If the noise reduction were linear, our image would change the same amount in each step, producing abrupt changes. A negatively sloped noise scheduler can remove large amounts of noise initially for faster progress, and then move on to less noise removal to fine-tune small details in the image.

Following the marble analogy, in the beginning it will probably be more useful to give it good hits and remove large chunks to advance quickly, while towards the end we will have to go very slowly to fine tune the details and not make an arm fall off.

A key aspect of the process is convergence. When a sampling algorithm reaches a point where more steps will not improve the result, the image is said to have converged.

Some algorithms converge very quickly and are ideal for testing ideas. Others take longer or require a greater number of steps but usually offer more quality. Others never do because they have no limit and offer more creativity.

With this article you will understand the nomenclature as well as the uses of the different methods without going into too much technical detail.

The image used in the demonstrations has been generated with the following parameters:

Checkpoint: dreamshaper_631BakedVae.safetensors. 检查点: dreamshaper_631BakedVae.safetensors 。
Positive prompt: ultra realistic 8k cg, picture-perfect black sports car, desert wasteland road, car drifting, tires churns up the dry earth beneath creating a magnificent sand dust cloud that billows outwards, nuclear mushroom cloud in the background far away, sunset, masterpiece, professional artwork, ultra high resolution, cinematic lighting, cinematic bloom, natural light.

正向提示: ultra realistic 8k cg, picture-perfect black sports car, desert wasteland road, car drifting, tires churns up the dry earth beneath creating a magnificent sand dust cloud that billows outwards, nuclear mushroom cloud in the background far away, sunset, masterpiece, professional artwork, ultra high resolution, cinematic lighting, cinematic bloom, natural light 。
Negative prompt: paintings, cartoon, anime, sketches, lowres, sun.

否定提示: paintings, cartoon, anime, sketches, lowres, sun 。
Width/Height: 512/512.
宽度/高度: 512 / 512 。
CFG Scale: 7. CFG 比例: 7 。
Seed: 1954306091. 种子: 1954306091 。
Samplers available 提供采样器
Depending on the software used you will find a varied list of possibilities. In this case we are going to analyze the samplers available in Automatic1111.

根据所使用的软件,您会发现各种可能性。在本例中,我们将分析 Automatic1111 中可用的采样器。

It is difficult to classify them into groups, although there are clearly two main approaches:

Probabilistic models such as DDPM, DDIM, PLMS and the DPM family of models. These generative models are able to generate an output based on the probability distribution estimated by the model. It would be like using a camera to photograph a landscape.
概率模型,例如 DDPM 、 DDIM 、 PLMS 和 DPM 系列模型。这些生成模型能够根据模型估计的概率分布生成输出。这就像用相机拍摄风景一样。

Numerical approach methods such as Euler, Heun and LMS. In each step, the solution to a particular mathematical problem is sought and the solution is estimated bit by bit. In this case it would be like using painting and a canvas to create the landscape and adding new details in each step.
数值逼近方法,例如 Euler 、 Heun 和 LMS 。在每个步骤中,都会寻求特定数学问题的解决方案,并一点一点地估计该解决方案。在这种情况下,这就像使用绘画和画布来创建风景并在每个步骤中添加新的细节。

—— 正文开始 ——


DDPM (paper) (Denoising Diffusion Probabilistic Models) is one of the first samplers available in Stable Diffusion. It is based on explicit probabilistic models to remove noise from an image. It requires a large number of steps to achieve a decent result.
DDPM (论文)(去噪扩散概率模型)是稳定扩散中最早可用的采样器之一。它基于显式概率模型来消除图像中的噪声。它需要大量的步骤才能获得不错的结果。

It is no longer available in Automatic1111.
它在Automatic1111 (国外的平台版本,不理解的话就当作时“秋叶的一键启动整合包就行了”)中不再可用。


DDIM (paper) (Denoising Diffusion Implicit Models) works similarly to DDPM, using in this case implicit probabilistic models. This difference produces better results in a much smaller number of steps, making it a faster sampler with little loss of quality.
DDIM (论文)(去噪扩散隐式模型)的工作原理与 DDPM 类似,在本例中使用隐式概率模型。这种差异可以用更少的步骤产生更好的结果,使其成为更快的采样器,而质量损失很小。

As can be seen in the cloud, better results are obtained with a high number of steps (100+). There are better alternatives as we will see below.


PLMS (paper) (Pseudo Linear Multi-Step) is an improvement over DDIM. Using a 50-step process it is possible to achieve higher quality than a 1000-step process in DDIM. Fascinating, isn't it? Well, read on, this is nothing.
PLMS (论文)(伪线性多步)是对 DDIM 的改进。在 DDIM 中,使用50步流程可以实现比1000步流程更高的质量。很迷人,不是吗?好吧,继续读下去,这没什么。

In the case of PLMS we cannot use few steps because it is not able to clean the noise, but between 50 and 100 steps it is already able to provide good results.
在 PLMS 的情况下,我们不能使用几个步骤,因为它无法消除噪声,但在 50 到 100 个步骤之间,它已经能够提供良好的结果。

Euler 欧拉

Euler is possibly the simplest method. Based on ordinary differential equations (ODE), this numerical method eliminates noise linearly in each step. Due to its simplicity it may not be as accurate as we would like but it is one of the fastest.
Euler 可能是最简单的方法。该数值方法基于常微分方程 (ODE),在每个步骤中线性消除噪声。由于其简单性,它可​​能不像我们希望的那么准确,但它是最快的之一。


Euler is so fast that it is able to deliver good results even in 10 steps. Its strong point is between 30 and 50 steps.
Euler 速度非常快,即使 10 步也能提供良好的结果。它的强项是30到50步之间。

Heun 赫恩

Heun is the perfectionist brother of Euler. While Euler only performs a linear approximation, Heun performs two tasks in each step, making it a second-order sampler. It first uses a linear approximation for prediction and then a nonlinear approximation for correction. This improvement in accuracy offers higher quality in exchange for a small drawback: it takes twice as long.
Heun 是 Euler 的完美主义者兄弟。 Euler 仅执行线性近似,而 Heun 在每个步骤中执行两个任务,使其成为二阶采样器。它首先使用线性近似进行预测,然后使用非线性近似进行校正。这种准确性的提高提供了更高的质量,但也带来了一个小缺点:需要两倍的时间。

Karl Heun developed this numerical method more than a century ago!
Karl Heun 在一个多世纪前开发了这种数值方法!

At 10 steps it still has some noise but it disappears in few more. As you can see, it offers high quality in 30 steps, although in 50 it offers a little more level of detail. In 100 steps it hardly changes the image and it is not worth getting old waiting for the result.
走了 10 步后,仍然有一些噪音,但再走几步就消失了。正如您所看到的,它在 30 个步骤中提供了高质量,尽管在 50 个步骤中它提供了更多的细节级别。在 100 个步骤中,它几乎不会改变图像,也不值得为了结果而老去。


LMS or Linear Multi-Step method is the cousin of PLMS that uses a numerical rather than a probabilistic approach (PLMS - P = LMS).
LMS 或线性多步方法是 PLMS 的近亲,它使用数值方法而不是概率方法 ( PLMS - P = LMS )。

Moreover, unlike Euler and Heun, it uses information from previous steps to reduce noise in each new step. It offers better accuracy in exchange for higher computational requirements (slower).
此外,与 Euler 和 Heun 不同,它使用先前步骤中的信息来减少每个新步骤中的噪音。它提供了更好的精度,以换取更高的计算要求(更慢)。

Using few steps we have a sampler capable of generating psychedelic images imitating the effect of drugs. Jokes aside, it is a sampler that is not worth it because despite being fast it needs around 100 steps to offer something decent.
通过几个步骤,我们就有了一个能够生成模仿药物效果的迷幻图像的采样器。抛开笑话不谈,这是一个不值得的采样器,因为尽管速度很快,但它需要大约 100 个步骤才能提供像样的东西。

Family of DPM models
DPM 模型系列

DPM (Diffusion Probabilistic Models) are probabilistic models that offers improvements over DDPM. Hence the similar name. There is also no implementation available in Automatic1111 because it has improved versions as we will see below.
DPM (扩散概率模型)是比 DDPM 提供改进的概率模型。因此有相似的名字。 Automatic1111 中也没有可用的实现,因为它有改进的版本,我们将在下面看到。

DPM2 is an improvement over DPM. You could say that it is version 2.
DPM2 是对 DPM 的改进。你可以说它是版本2。


With 10 steps you already get an impressive quality (don't try 5 steps, you won't like the result). Around 30 to 50 steps is the ideal point. More steps are usually not worth it.
通过 10 个步骤,您已经获得了令人印象深刻的质量(不要尝试 5 个步骤,您不会喜欢结果)。大约 30 到 50 步是理想的点。更多的步骤通常是不值得的。

On the other hand we have DPM++, which is also an improvement of DPM. It uses a hybrid approach combining deterministic and probabilistic methods for sampling and subsequent noise reduction. There is no basic implementation of this sampler in Automatic1111, but it is combined with other methods. We will see this in the next section.
另一方面,我们有 DPM++ ,它也是 DPM 的改进。它使用结合确定性和概率方法的混合方法进行采样和随后的降噪。在Automatic1111中没有这个采样器的基本实现,但是它与其他方法结合在一起。我们将在下一节中看到这一点。

Thus, two versions with improvements were born from DPM: DPM2 and DPM++.
因此,从 DPM 诞生了两个改进的版本: DPM2 和 DPM++ 

Faster DPM models (DPM-Solver and UniPC)
更快的 DPM 模型( DPM-Solver 和 UniPC )
Diffusion Probabilistic Models (DPM) are, as the name suggests, probabilistic. In each step, equations are not solved by deterministic numerical methods as in the case of Euler, Heun or LMS, but the problem is approached by approximation to try to sample as accurately as possible.
顾名思义,扩散概率模型 ( DPM ) 是概率性的。在每个步骤中,方程并不是像 Euler 、 Heun 或 LMS 那样通过确定性数值方法求解,而是通过近似来尝试解决问题尽可能准确地采样。

Within these models there is a piece called solver, an algorithm that has an important role in calculating and approximating a probability distribution in sampling. And this is where a new technique known as DPM-Solver is implemented that shortens the duration of each step.
在这些模型中,有一个称为求解器的算法,该算法在计算和近似采样中的概率分布方面发挥着重要作用。这就是一种名为 DPM-Solver 的新技术的实现,它可以缩短每个步骤的持续时间。

In other words, models like DPM fast (paper) or DPM++ 2S/DPM++ 2M (paper) implement a faster solver that will save time in the sampling process.
换句话说,像 DPM fast (论文)或 DPM++ 2S / DPM++ 2M (论文)这样的模型实现了更快的求解器,可以节省采样过程的时间。

DPM fast

It will be fast (and not that fast either), but using few steps it is unusable. Interestingly it offers a different result to the rest of samplers and it seems that the cinematic effect is more pronounced.

In the case of DPM++ 2S/DPM++ 2M the number 2 means that they are second order. That is, they use both a predictor and a corrector to approximate the result accurately.
在 DPM++ 2S / DPM++ 2M 的情况下,数字 2 表示它们是二阶的。也就是说,他们同时使用预测器和校正器来准确地近似结果。

The S stands for Single step. A single calculation is performed in each step, so it is faster.
S 代表 Single step 。每一步执行一次计算,因此速度更快。

In contrast, the letter M stands for Multi step, an approach in which multiple calculations are performed in each step, taking into account information obtained in previous steps. This equates to more accurate and higher quality convergence at the cost of taking longer.
相反,字母 M 代表 Multi step ,这是一种在每个步骤中执行多次计算的方法,同时考虑到先前步骤中获得的信息。这相当于更准确、更高质量的收敛,但代价是花费更长的时间。

In both modalities this solver is faster than the default DPM model solver.
在这两种模式中,该求解器都比默认的 DPM 模型求解器更快。

There is no Automatic1111 implementation of DPM++ 2S, only with A, Karras and SDE variants (more on that later). So let's see some samples of DPM++ 2M.
DPM++ 2S 没有 Automatic1111 实现,只有 A 、 Karras 和 SDE 变体(稍后会详细介绍)。让我们看一下 DPM++ 2M 的一些示例。

Little to say about this all-rounder sampler. It offers impressive results in 30 steps and if you give it some more time it can be squeezed even more.
关于这款全能采样器没什么可说的。它只需 30 个步骤即可提供令人印象深刻的结果,如果您多给它一些时间,它可以压缩得更多。

As for UniPC (paper), it is a solver that consists of two parts: a unified predictor (UniP) and a unified corrector (UniC). This method can be applied to any DPM model and focuses on delivering the maximum possible sampling quality in the least amount of steps. Remember now when PLMS brought down to 50 steps what DDIM did in 1000? Well, in some cases UniPC is able to generate quality images in as few as 5 or 10 steps.
至于 UniPC (论文),它是一个由两部分组成的求解器:统一预测器( UniP )和统一校正器( UniC )。此方法可应用于任何 DPM 模型,并专注于以最少的步骤提供尽可能高的采样质量。现在还记得 PLMS 将 DDIM 减少到 50 步时的 1000 步吗?嗯,在某些情况下, UniPC 能够在短短 5 或 10 个步骤内生成高质量图像。

Thus, UniPC can be integrated in DPM models both Single step and Multi step, making it comparable to DPM++ 2S or DPM++ 2M, with the particularity of offering better results when the number of steps is very low.
因此, UniPC 可以集成到 DPM 模型 Single step 和 Multi step 中,使其与 DPM++ 2S 或 DPM++ 2M ,其特点是在步骤数非常少时提供更好的结果。

Even the UniC corrector can be integrated into these sampling algorithms to achieve higher efficiency (e.g. DPM++ 2S + UniC).
甚至 UniC 校正器也可以集成到这些采样算法中以实现更高的效率(例如 DPM++ 2S + UniC )。

UniPC 统一电脑

In this example 10 steps is not enough to generate an image without noise, but in 15 or 20 you will get it. In 30 steps it is magnificent and there is no need to go any further, although there is still some room for improvement.
在此示例中,10 个步骤不足以生成没有噪声的图像,但在 15 或 20 步内您就会得到它。三十级台阶已经很宏伟了,没有必要再进一步了,尽管还有一些改进的空间。

More accurate DPM models (Adaptive)
更准确的 DPM 模型 ( Adaptive )
The DPM adaptive model is an extension of the DPM model in which it adapts the step size according to the difficulty of the problem is trying to solve.
DPM adaptive 模型是 DPM 模型的扩展,它根据要解决的问题的难度来调整步长。

In other words, it is as if the specified number of steps is ignored and the algorithm is free to sample more efficiently until the best possible convergence is achieved. It generates higher quality images at the expense of taking as long as it needs to (it is the slowest sampler).

DPM adaptive DPM自适应

In this case it has taken triple or quadruple the time with respect to other samplers but the result is amazing. The image composition is different from all samplers and is more like DPM fast.
在这种情况下,与其他采样器相比,它花费了三倍或四倍的时间,但结果是惊人的。图像组成与所有采样器都不同,更像 DPM fast 。

Other features 其他特性

Only one sampling algorithm can be chosen. Either Euler or DPM can be used, but not both at the same time. Instead, when we talk about variants or extra features, these can be combined.
只能选择一种采样算法。可以使用 Euler 或 DPM ,但不能同时使用两者。相反,当我们谈论变体或额外功能时,可以将它们组合起来。

For example, we can use the sampler named DPM2 A Karras. Let's see what these new values mean.
例如,我们可以使用名为 DPM2 A Karras 的采样器。让我们看看这些新值意味着什么。

Ancestral variants 祖先的变种

When a sampler contains the letter A, it usually means that it belongs to the category of ancestral variants. These variants add, in each new step, random variables obtained from previous steps. It is as if after cleaning up the noise in one step, some of the previous noise is added back.
当采样器包含字母 A 时,通常意味着它属于祖先变体类别。这些变体在每个新步骤中添加从先前步骤获得的随机变量。就好像一步清理掉噪音后,之前的一些噪音又加回来了。

Samplers with this feature never converge because of the random noise added in each step. If there is always noise to remove, you can always go one step further.

This makes them more creative samplers. An extra step does not necessarily increase the quality, but rather gives another similar result.

If you try to reproduce an image generated with Stable Diffusion and you don't succeed even though you are using the same seed and the same parameters, it may be because you are using an ancestral sampler. This is normal! The noise that is re-added in each step is random and different implementations or versions of the sampler will almost certainly generate different results.

Some examples of samplers are Euler A, DPM2 A or DPM++ 2S A.
采样器的一些示例是 Euler A 、 DPM2 A 或 DPM++ 2S A 。

Euler A 欧拉A

Euler A gives a great result in 25-30 steps being also very fast. In 50 steps the quality is worse and then in 100 steps it is better again. It is a lottery. Moreover, you can see how the image composition is constantly changing due to the random noise introduced in each step. Far from being a drawback it is perhaps its greatest advantage.
Euler A 在 25-30 步中给出了很好的结果,而且速度也非常快。在 50 个步骤中,质量会变差,然后在 100 个步骤中,质量又会更好。这是彩票。此外,您还可以看到图像组成如何由于每个步骤中引入的随机噪声而不断变化。这绝不是一个缺点,而可能是它最大的优点。

Karras Variants 卡拉斯变体

Variants with the word Karras (or K) (paper) refer to work led by Nvidia engineer Tero Karras. This process introduces a series of improvements in some samplers, achieving improved efficiency in both the quality of the output and the computation required for sampling.
带有单词 Karras (或 K )(论文)的变体指的是 Nvidia 工程师 Tero Karras 领导的工作。此过程在一些采样器中引入了一系列改进,从而提高了输出质量和采样所需计算的效率。

Some samplers using these changes are: LMS Karras, DPM2 Karras, DPM2 A Karras, DPM++ 2S A Karras, DPM++ 2M Karras or DPM++ SDE Karras.
使用这些更改的一些采样器是: LMS Karras 、 DPM2 Karras 、 DPM2 A Karras 、 DPM++ 2S A Karras 、 DPM++ 2M Karras 或 DPM++ SDE Karras 。

Like DPM++ 2M, this sampler offers very good results between 30 and 50 steps, but the Karras version has the advantage of offering better results in a reduced number of steps as can be seen in the following example:
与 DPM++ 2M 一样,此采样器在 30 到 50 个步骤之间提供了非常好的结果,但 Karras 版本的优点是可以在减少步骤数的情况下提供更好的结果,如在以下示例:

If you use a high number of steps you will have a hard time seeing the difference.

Stochastic variants 随机变量

The SDE (paper) variants use stochastic differential equations. Without going into further detail, using this type of differential equations allows to model the noise in a more sophisticated and accurate way, using information from previous steps, which in principle would generate higher quality images in exchange for being slower. Being stochastic, it never converges, so the higher the number of steps they do not offer higher quality, but rather more variations, just like ancestral samplers.
SDE (论文)变体使用随机微分方程。无需进一步详细说明,使用这种类型的微分方程可以使用先前步骤中的信息,以更复杂和更准确的方式对噪声进行建模,原则上这会生成更高质量的图像,以换取更慢的速度。由于是随机的,它永远不会收敛,因此步数越高,它们并不能提供更高的质量,而是提供更多的变化,就像祖先的采样器一样。

At the date of publication of this article we have DPM++ SDE, DPM++ 2M SDE, DPM++ SDE Karras and DPM++ 2M SDE Karras.
在本文发布之日,我们有 DPM++ SDE 、 DPM++ 2M SDE 、 DPM++ SDE Karras 和 DPM++ 2M SDE Karras 。


Stochastic samplers are slow but offer incredible results even with 10 steps. Their results are also more varied and creative. As they never converge they are an alternative to ancestral samplers.

What is the best sampler in Stable Diffusion?

Is a Ferrari or a Jeep better? Well it depends on whether you're going off-road, doesn't it?

Depending on what you need it's better to use one type of sampler or another. With the above information I hope it will be easy to choose, but here are some hints.


Image quality 画面质量

If you are looking for quality it is a good idea to pursue convergence. That's the point at which you get the highest quality. If you don't want to sacrifice too much generation speed, forget about samplers like DDIM that need hundreds of steps to converge. Heun and LMS Karras offer good results but it is better to use DPM++ 2M or its Karras version.
如果您追求质量,那么追求融合是一个好主意。这就是您获得最高质量的点。如果您不想牺牲太多生成速度,请忘记像 DDIM 这样需要数百步才能收敛的采样器。 Heun 和 LMS Karras 提供了良好的结果,但最好使用 DPM++ 2M 或其 Karras 版本。

You can also try DPM adaptive if you are not in a hurry, or UniPC if you are.
如果您不赶时间,也可以尝试 DPM adaptive ,如果赶时间,也可以尝试 UniPC 。

With these samplers mentioned above you will get good results in 20-30 steps although it doesn't hurt to try a few extra steps.
使用上面提到的这些采样器,您将在 20-30 个步骤内获得良好的结果,尽管尝试一些额外的步骤也没有坏处。


Generation speed 生成速度

If you are testing prompts you don't want to spend so much time waiting for results. In this case and where you are not looking for maximum quality but to test changes quickly I recommend using DPM++ 2M or UniPC with a small number of steps.
如果您正在测试提示,您不想花费太多时间等待结果。在这种情况下,如果您不寻求最高质量,而是为了快速测试更改,我建议使用 DPM++ 2M 或 UniPC 以及少量步骤。

With just 10-15 steps you will get a very decent image.
只需 10-15 个步骤,您就可以获得非常不错的图像。

If you don't care about reproducibility you also have Euler A, a fast and good quality ancestral sampler. My favorite sampler!
如果您不关心再现性,您还可以使用 Euler A ,这是一个快速且优质的祖先采样器。我最喜欢的采样器!

Creativity and flexibility

This section is reserved exclusively for ancestral and stochastic samplers. They don't offer bad quality nor are they slow, they are just different.

The problem or advantage (depending on how you look at it) of these samplers is that if you have an image generated in 40 steps, having done it in 50 steps can make the image better or worse. You will have to test continuously. And this lottery makes these samplers more creative since you can always change the number of steps to obtain small variations.
这些采样器的问题或优点(取决于您如何看待它)是,如果您有 40 个步骤生成的图像,则在 50 个步骤中完成可能会使图像变得更好或更差。你必须不断地进行测试。这种抽奖使这些采样器更具创造性,因为您始终可以更改步骤数以获得小的变化。

Of course here Euler A and DPM++ SDE Karras stand out. Try generating images in 15 steps, 20 steps, 25 steps... and see how the result changes.
当然,这里 Euler A 和 DPM++ SDE Karras 很突出。尝试用 15 步、20 步、25 步......生成图像,看看结果如何变化。


版权声明:本文采用知识共享 署名4.0国际许可协议 [BY-NC-SA] 进行授权
文章名称:《【中英双字】一篇文章教你怎么选择Stable Diffusion扩散采样器的类型以及各个扩散器的功能详解教学,再也不用担心不知道怎么选》

评论 抢沙发


致力于发现ai人工智能应用的新世界, 长期更新目前热门AI教程与动态!期待在这个工业变革时代中,希望你我都能发出一点光。