Mastering AI Image Control with Shaker’s New Control Net Model

if you have already subscribed to any previous workflows, you should already have this one, check your email.

If you found yourself needing to have a little bit more control over your AI-generated images, you’re not alone. Until recently, we’ve only had one major Control Net model released for Flux, initially by XLabs. However, Shaker, in combination with InstantX, has just released a new Control Net model that takes things to the next level. This new model includes everything from OpenPose, Kanye, Depth, and Tiled, far surpassing what we had from XLabs. Best of all, it’s packed into a single model, making installation a breeze. Let’s dive right in and explore this tool in detail.

Installation and Setup

For those interested in trying out Shaker’s new Control Net model, you’ll find all the necessary links in the description below. You can download the model from the Chakra Labs page on Hugging Face. It’s called Flux One Dev Control Net Union Pro, showcasing methods like Canny, OpenPose, and Depth.

Step-by-Step Guide:

1. Downloading the Model:
   • Navigate to the files and versions section.
   • Download the model into your Control Net folder in your ComfyUI directory. For better organization, consider creating a separate folder named Flux within the Control Net folder.
2. Renaming the Model:
   • The downloaded model will have a generic name, DiffusionPyTorchModel, which can be confusing. Rename it to something more descriptive, like ControlNet_Union_SaveTensors.
3. Loading in ComfyUI:
   • Head over to ComfyUI and load your standard Flux workflow. It’s essential to note that if you’re using a 24-gigabyte GPU like a 3090 or 4090, you may encounter memory issues. For such cases, a quantized version of Flux is recommended, which can be downloaded using the link in the description. The Q8 model is generally preferred.

Practical Usage

Now that we’ve set everything up, let’s see how it performs in practice:

1. Loading the Model:
   • Use the ComfyUI custom nodes manager to download ComfyUI_gguf, enabling you to load the quantized model.
2. Control Net Adjustment:
   • Load the unit loader gguf and substitute your load diffusion model with the unit loader. Ensure you set the Control Net type to match your specific needs.

Key Parameters:

• Strength: Determines how intensely the Control Net is applied to your image.
• Start Percent & End Percent: Defines at what points during the image generation process the Control Net is active.

Hands-On Example

Imagine generating an image of a meditating character. Start by selecting the OpenPose control type for the Control Net. Utilize prompts to specify elements like “fantasy character, girl with freckles, blue hair, long ears,” and add in specifics such as “meditating in a street with a town square in the background.”

• Initial Generation:
  • If you have issues such as poorly rendered hands, adjust the End Percent to give the Flux model more room to refine the image. You might find a setting of around 0.50 to be more effective.
• Alternate Control Net Options:
  • If OpenPose doesn’t give you the desired outcome, try switching to normal or auto types. Depth and CANI control nets can sometimes yield better results, as seen in examples with various styles and poses.

Addressing Common Issues

Hands can sometimes appear irregular using certain Control Nets. To optimize results:

• Test different settings for Strength and End Percent.
• Experiment with various Control Net types.
• Use preprocessors to fine-tune elements like CANI or Depth maps.

Conclusion

Shaker’s new Control Net model greatly enhances the capabilities offered by Flux. With features like OpenPose, Kanye, and Depth all packed into one model, you can achieve professional-quality AI-generated images with ease. For advanced workflows and free access to templates, consider checking out our Patreon.

Additional Resources

• PromCrafters: Their database offers an excellent starting point for project prompts and can refine and expedite the creative process.
• Exclusive Content: Join our Patreon for a more comprehensive version of these workflows, along with additional poses and customization options.

Finally, don’t forget to like and subscribe to stay updated with the latest tips and tutorials. Happy creating, and I can’t wait to see what you come up with!