Get an AI Consultation

Unveiling the Game-Changing ComfyUI Update

Posted by:

Trizia Fortich

|

On:

|

Introduction

Nvidia has recently introduced a groundbreaking paper titled “Align Your Steps,” revolutionizing stable diffusion outputs. This paper delves into the intricacies of the sampling process involved in image creation using stable diffusion techniques. The outcome? A significant enhancement in image quality achieved in half the usual number of steps. The beauty of this advancement lies in its compatibility with existing models, eliminating the need for new model iterations.

Understanding “Align Your Steps” by Nvidia

In a nutshell, “Align Your Steps” focuses on optimizing the sampling process at each step of image generation. By fine-tuning this process, Nvidia has achieved remarkable improvements in image fidelity and detail. The paper showcases several examples comparing results from Stable Diffusion 1.5 with and without the “Align Your Steps” technique. The visual disparities are striking, with images generated using the new method exhibiting superior quality and adherence to prompts.

Key Highlights from the Paper

  1. Improved Fidelity: Comparisons between images generated with and without the “Align Your Steps” technique reveal a noticeable enhancement in fidelity and detail.

2. Prompt Adherence: The technique significantly enhances prompt adherence, ensuring that generated images closely align with given prompts.

Tiger photos adhere to instructions, accurately wearing construction gear provided.

3. Reduction of Artifacting: Notably, the paper addresses artifacting issues common in stable diffusion outputs, resulting in cleaner and more refined images.

Illustrations of Studio Ghibli houses.

Implementing “Align Your Steps” in Comfy UI

Now, let’s explore how you can leverage this groundbreaking technique in Comfy UI. Here’s a step-by-step guide:

  1. Update Comfy UI: Ensure that your Comfy UI is updated to the latest version, where the “Align Your Steps” feature has been integrated.

2. Swap Samplers: Replace the existing KSampler with the Sampler Custom Node. Navigate to Add Node > Sampling > Custom Sampling > Schedulers > Sampler Custom.

3. Configure Parameters: Adjust the parameters, including the model type and number of steps, within the Align Your Steps Scheduler node. Select the appropriate model type based on your requirements and set the desired number of steps. Nvidia recommends starting with 10 steps for SDXL models, with the option to increase for further experimentation.

Setting Up the Workflow

1. Access the workflow available on the website, which includes both the Sampler Custom and regular KSampler setups.

2. Ensure both samplers use the same seed, fixed LATENT, and positive and negative prompts for controlled parameters.

3. This setup ensures a realistic comparison between the outputs of both samplers.

Comparing Outputs:

  1. Start with the text prompt “beautiful woman standing on a spaceship.”

2. Compare the outputs.

Comparison between standard KSampler output (on the right) at 15 steps, and Align Your Steps output (on the left).

3. Notice the tremendous difference in quality, with AYS (Align Your Steps)  producing a significantly superior image with fewer artifacts.

 Handling Artifacts and Improving Quality:

  1. For situations where AYS doesn’t perfectly render faces, utilize a face detailer and Upscaler for further refinement.

Final image with an upscale, all generated with 15 steps.

Caveats and Compatibility:

  1. Understand that AYS (Align Your Steps)  is tailored for DPM++ samplers; using it with other samplers may yield unpredictable results.
  2. Ancestral samplers are not optimized for AYS and may exhibit compatibility issues.

Comparing KSampler and Align Your Steps

1. Align Your Steps VS. KSampler: We observed that while the base KSampler seemingly adhered slightly better to the provided prompt, with the character holding a glass resembling a martini more closely, the AYS interpretation offered an intriguing space-themed alternative. Despite this, the AYS image consistently exhibited superior fidelity and overall quality.

2. Exploring Further Possibilities Our exploration ventured into pushing the boundaries of image generation by testing scenarios involving characters holding multiple items. Unexpectedly, the vanilla KSampler momentarily outperformed AYS, contrary to previous observations. However, upon closer inspection, even with increased KSampler steps, the AYS output maintained superior quality, particularly evident in finer details such as hand positioning and artifacting reduction.

Text Prompt: beautiful woman standing on a spaceship, holding a glass space martini in the left hand and a space gun in the right hand.

4. Compare how both samplers interpret the prompts, highlighting differences in output quality and adherence. The impact of prompt detailing on model performance became apparent as we fine-tuned prompts to elucidate model discrepancies. Despite both models featuring the gun and martini elements, the AYS interpretation showcased a marked improvement in the character’s interaction with these items, lending a more naturalistic feel to the scene.

Testing Unique Scenarios:

  1. Experiment with unconventional prompts like “a tiger having a cup of coffee in a coffee shop, wearing a top hat and a monocle.”
  2. Compare outputs for sharpness, detail, and overall quality between AYS and regular KSampler.

Text Prompt: a tiger having a cup of coffee in a coffee shop. The tiger is wearing a (top hat and a monocle.)

Here are the outputs from both samplers. Although both images are quite similar and neither tiger is depicted wearing a monocle, there are noticeable differences in quality. The AYS version exhibits significantly higher sharpness and detail compared to the vanilla KSampler version. This is particularly evident in the eyes, where the AYS image reveals finer details and even includes a catch light, making the eyes appear more lifelike. In contrast, the eyes in the vanilla KSampler image appear more muted and lack the same level of detail and vibrancy.

What did you think? Is AYS (Align Your Steps)  going to revolutionize your workflows?

AYS is an exciting addition to workflows, allowing the generation of high-quality images in half the time while still utilizing favorite models. For those using AYS (Align Your Steps), joining our growing Discord community is highly recommended. It’s a fantastic place to see what everyone is creating and share work in the ‘#share-your-work’ section. The community is expanding quickly, and new members are always welcome. Plus, Discord is the perfect platform to ask questions if there’s anything unclear about Stable Diffusion, especially since images can’t be attached in the comments below.

If these tutorials are found helpful, please consider supporting us on Patreon. A basic version of the workflow is available to everyone on the website. However, for the full version, which includes face replacement, upscaling, and video features, check out the Patreon page. Supporting us on Patreon not only helps sustain the channel but also provides additional perks and exclusive content.

Thank you for your support, and see you in the next tutorial!

Join the Community: