Video assisted by artificial intelligence continues to progress and offers ever more astonishing possibilities, such as transforming a video into a cartoon. Or even animate and make a photograph speak.
L’applicationapplication Gen from Runway ML was one of the pioneers of prompt-based video generation. And the Gen-3 version introduced a rather remarkable function: the possibility of transforming a video by improving the character, the settings, by changing theatmosphereatmosphere…
What is Runway?
Runway ML is a true toolbox of artificial intelligence services related to image processing. Generating videos from a prompt is obviously the basic tool. But there are also a large number of effective services: removing the background from a video, blurring a face, generating subtitles, etc.
Runway only has a limited offer when it comes to free. You are entitled to 125 credits, which corresponds to approximately 10 seconds of video and therefore, if this service appeals to you, you will have to subscribe to a paid offer, i.e. at least 12 dollars per month. And if you have very high needs, it is possible to use these services unlimitedly by paying $76 per month. You can unsubscribe at any time.
Video to video
In this case, the service that will interest us is “Video to Video”. In other words, transforming a video according to another model.
On the Runway website, click Get started. Select “Video to Video” and make sure you select “Gen-3”. Then upload a video from your Mac or PC. Once the video is loaded, click on “Samples”. You will then see a series of predefined models appear: 3D Cartoon, Claymation (modelling clay animation), Dramatic, Glass (glass)… You can select the desired effect and click on “Apply”. A prompt is provided and you are free to enrich it (in English). You can also imagine a totally original prompt.
Then click on “Generate”. The generation starts and usually takes several minutes. Please note that only 10 seconds of video can be taken into account.
As you can see, the converted video faithfully respects the movementsmovements and even the look of the original. However, the decor and atmosphere are completely transformed. Here are some examples we have accomplished.
Some examples of transformations with Gen 3. © D.Ichbiah with Gen-3 from RunwayML
Complete transformation
There is an alternative in Gen3. In the left column, you can choose “Act One”, select your clip (by drawing it from the Assets) and completely transform the character of your video by choosing one of the models offered – the lip movements will be respected, but the video will be radically different from the original.
