What we are working on: Generative AI Meets Video Editing

With generative AI on everyone's lips, we would like to draw attention to some of the topics we are working on regarding the matter.

We would like to introduce you to a work-in-progress feature we are dubbing “The AutoCut”. It uses a large language model to find relevant moments along with statements within a video.

Although this in and of itself may not necessarily cause a wow effect, the magic happens when integrating this algorithm into a tool chain consisting of a search portal for media professionals such as (MediaPortal) coupled together with a browser-based video editor (VidiEditor). Employed in this context, it would allow a video’s most relevant sequences to be edited in no time.

A typical workflow would involve a media professional browsing and picking their media through MediaPortal’s refined search functions.

On every media item, in this case a video, the user can start configured workflows, which in this case is a workflow called “OpenAI AutoCut”. The user can then specify how many relevant moments they would like to have proposed.

Additionally, the user can define related term(s) to add intuition to the algorithm to focus on. Furthermore, the language of the reasoning is configurable. This makes it particularly interesting if the video’s contained language is not spoken.

As shown below, all relevant and time accurate moments of the video – a press conference in this case – can be viewed and navigated in MediaPortal as well.

And here comes the benefit of automation. Handing over these snippets to an integrated VideoEditor works like a charm. When carried over these are linked after another, segment after segment to create a pre-cut timeline along with all of the metadata generated by the AutoCut.