NVIDIA’s Magic 1-for-1 AI model, developed in collaboration with researchers from Peking University and Hedra Inc., can generate high-quality one minute clips within 60-seconds. It achieves this by factorizing the text-to-video generation task into two separate easier tasks for diffusion step distillation, namely text-to-image generation and image-to-video generation.
Magic 1-For-1 leverages an advanced step distillation algorithm to train a “generator” model that can produce high-quality videos in just a few steps. First, the generator model is trained in conjunction with two other models: one that approximates the real data distribution and another that approximates the distribution of the generated data. By combining these distributions, it learns to produce realistic videos efficiently. You can download it here now.
No products found.
The model further enhances efficiency through techniques like adversarial step distillation, which accelerates the generation process, and parameter sparsification, which reduces memory usage. These methods allow the model to generate short video clips in just a few seconds. By using a sliding window approach during inference, the model can also create longer videos, up to a minute in length, while maintaining good visual quality and motion,” said the researchers.
