Stability AI, an AI startup, has announced a new AI model called Stable Video Diffusion. This model uses AI technology to generate videos by animating existing images. Unlike other video-generating models, Stable Video Diffusion is available in open source and commercially. However, it is currently in a research preview stage.
Key Takeaway:
Stability AI has introduced Stable Video Diffusion, an AI model that uses existing images to generate videos. While it is currently in a research preview stage, the model shows promise in terms of quality and extensibility. However, there are concerns about potential misuse, and it remains to be seen how Stability AI will address legal and ethical challenges. Nevertheless, the company has plans for further development and aims to commercialize Stable Video Diffusion for various industries.
Terms of Use and Intended Applications
Those who wish to use the Stable Video Diffusion model must agree to certain terms of use. These terms outline the intended applications, such as educational or creative tools, design, and other artistic processes. On the other hand, the terms also specify the non-intended applications, such as factual or true representations of people or events.
While Stability AI has taken precautions with the terms of use, there is still concern about the potential misuse of Stable Video Diffusion. Past instances of AI research previews have resulted in the circulation of models on the dark web, leading to the creation of nonconsensual deepfake porn and other malicious uses.
The Functionality of Stable Video Diffusion
Stable Video Diffusion consists of two models: SVD and SVD-XT. SVD can transform still images into 576×1024 videos with 14 frames, while SVD-XT increases the frames to 24. Both models can generate videos at a speed of 3 to 30 frames per second.
In terms of quality, Stable Video Diffusion produces high-quality four-second clips. Comparison with other video generation models from Meta, Google, Runway, and Pika Labs shows that Stability AI’s models hold their own.
Limitations and Future Plans
Although Stable Video Diffusion showcases impressive capabilities, it does have its limitations. The models cannot generate videos without motion or slow camera pans, be controlled by text, accurately render text, or consistently generate faces and people. However, Stability AI acknowledges these limitations and emphasizes the extensibility of the models for various use cases, including the generation of 360-degree views of objects.
Looking ahead, Stability AI has plans to expand the capabilities of Stable Video Diffusion. The company is developing additional models that will build on and extend the functionality of SVD and SVD-XT. They are also working on a “text-to-video” tool to enhance the models’ capabilities for the web. The ultimate goal is commercialization, with potential applications in advertising, education, entertainment, and more.
