Introduction
AI ToonCrafter is not just another tool; it’s a glimpse into the future of animation. With its release, we’re entering an era where low-quality animations become a thing of the past. This cutting-edge tool utilizes advanced interpolation techniques to seamlessly generate frames, resulting in animations of unparalleled quality.
One of the most exciting aspects of AI ToonCrafter is its accessibility. Unlike some high-end animation software that demands powerful hardware, AI ToonCrafter can run on your own computer. This means that regardless of your setup, you can unleash your creativity and explore the endless possibilities of animation.
Understanding AI ToonCrafter
The VRAM requirement is roughly 26 GB, which is too high for cards like the 3090 or 4090. But there’s a solution for any machine.
This tool does advanced interpolation, creating frames between starting and ending frames. It comes with examples to try. We’ll check those out.
In this short tutorial, I’m using an A40 with 48 GB VRAM. Making a short animation takes about 40 seconds. Comparing it with similar tools, you’ll see the difference in coherence and animation quality.
And to really demonstrate its capabilities, I didn’t just rely on the pre-made examples. I decided to create my own AI image using a tool I like called ‘Scenario’. I took a female character and made her blink by redrawing her eyes.
After feeding those images in, here’s what came out. It’s straightforward, yet quite impactful.
Note: There’s no ComfyUI node yet, but it’s in the works. Once it’s out, I’ll make a tutorial. This could revolutionize animation.
How to Install and Use ToonCrafter on RunPod
Step 1: Create a RunPod Account
If a Runpod account is not yet created, use the affiliate link: https://runpod.io?ref=49tc28ho. This supports the creation of more content and experiments.
- Log in to Runpod:
- Navigate to the Runpod website.
- Log in to the account.
- Deploy a Pod:
Head to the “Deploy” section.
Select a GPU with at least 48 GB of VRAM. The A40 at $0.69 per hour is a recommended option.
Choose the PyTorch 2.20 template for optimal compatibility.
Once your pod is running:
Connect to Jupyter Lab: Your pod will default to JupyterLab. Connect to it and open an empty workspace.
Open Terminal: Open a terminal within Jupyter Lab.
Clone the ToonCrafter Git
Get the ToonCrafter files from their Git, then get the web link. Go back to your lab and copy the link to get the files.
Once you’ve downloaded ToonCrafter onto your computer or Jupyter lab, you’ll find a folder named ToonCrafter. Navigate into that folder by using the command CD ToonCrafter. Once inside, run an LS or DEER command to ensure that all the necessary files are present.
If you’re following the instructions, they typically advise creating and activating an Anaconda environment on Linux, Windows, or Mac machines. I’d suggest proceeding with this if you’re working on any of these platforms. However, if you’re using RunPod, you can skip this step. RunPod operates within an isolated environment, meaning everything you set up and install here is specific to this instance only.
Go ahead and grab the ‘pip install -r requirements. txt” now;
You may encounter some errors during the installation process, but I’ve found ways to ignore them and proceed smoothly. While the installation is ongoing, let’s move on to obtaining the required model.
Returning to the git, the model is specified there, and I’ll put a link again in the description below for everyone’s convenience. Head over to Hugging Face, where you’ll find the page with the download link.
Right-click to copy the link, then return to your Jupyter Lab and wait for the installation to finish.
Alright, we’re all set. The installation is complete. As mentioned earlier, you might come across an error regarding a dependency conflict. However, I managed to bypass it and proceed smoothly since this environment is designed for PyTorch 2.2.0. It’s perplexing why it assumes we’re running PyTorch 2 only, but thankfully, it didn’t hinder our progress.
Now that ToonCrafter is installed, let’s move on to downloading the model we copied earlier. Navigate to the ToonCrafter folder—oops, no need to CD into it again since we’re already there. However, we do need to create a checkpoint directory. So, let’s create one by typing ‘mkdir checkpoints’ and then ‘cd checkpoints’. Next, create the directory ‘tooncrafter_512_interp_V1’. Once that’s done, CD into that directory. Finally, use ‘wget’ to download the model. Simply paste the URL we obtained from Hugging Face: ‘https://huggingface.co/Doubbi/ToonCrafter/resolve/main/model.ckpt’.
Once you’ve located that on your side panel, navigate into the ToonCrafter folder and locate the file named ‘gradio_app.py’. Double-click on that file and scroll down to the bottom where you’ll find the last line, which reads ‘dynamiccrafter_iface.Launch.’ Now, let’s make a couple of additions there.
First, include ‘share=True, auth(“user,”pass”)—make sure it’s capitalized, or it won’t be recognized. Then, insert two colons, two commas, and two colons.
In each of the commas, enter a username and password. I typically use “user” for the username and something more secure. Don’t forget to add a comma at the end. Save the file, and you’re all set to proceed!
After the installation is complete, navigate back to your ToonCrafter base folder by using the command ‘cd’ or ‘cd ..’. Once you’re there and you’ve made the required modifications to the gradio app file, simply type in ‘python gradio_ app.py’ and you’re good to go!
Upon initialization, the app will provide you with two URLs: one for local use, typically 127.0.0.1 with the necessary port, and another for public access, suitable for RunPod usage. If it appears to be hanging, don’t worry; this is normal as it takes a moment to warm up.
Once it’s up and running, you’ll see two URLs: one for local use and the other for public access. If you’re using RunPod, go for the public URL. If you have a powerful GPU on your machine, you can choose the local URL instead.
Enter the username and password you’ve created.
And that’s it! You now have it installed and ready to go.
Conclusion
The installation process for ToonCrafter is straightforward, especially if you’re following along with our step-by-step guide. Despite encountering a few minor hiccups along the way, such as dependency conflicts, we were able to overcome them and successfully set up ToonCrafter on our system.
Moving forward, I’ll continue to explore ToonCrafter and its capabilities extensively in the coming days. I’ll be sharing my findings and results on Discord and my Instagram, so be sure to follow me there to stay updated. Additionally, I encourage you to join our Discord community and share your own experiences and creations with ToonCrafter. Your input and contributions are invaluable, and I’m excited to see what you can achieve with this powerful tool.
If you have any questions or need further assistance, don’t hesitate to reach out. Let’s unleash our creativity with ToonCrafter together!