Google has unveiled a new AI video generator called Lumiere

[or1k]

Google has unveiled a new AI video generator called Lumiere, which is based on a spatio-temporal diffusion model for realistic video generation. It is capable of generating dynamic images from a textual description and is arguably the most powerful to date.

A unique feature of Google Lumiere is its ability to generate video in a single pass, unlike other models that work with multiple keyframes and then interpolate over time. Lumiere can convert text to video, convert static images to dynamic images, create videos in a specific style, edit existing video based on written prompts, and animate certain areas of a static image.

The model is trained on a dataset of 30 million video clips with text descriptions. The output is 5-second clips with a resolution of 1024 × 1024 pixels.

Google recognizes the threat posed by such projects and emphasizes the importance of developing tools to detect bias and malicious use of the technology to ensure the model runs safely.