The last few years have seen tremendous growth in what AI is capable of achieving.  This is certainly the case in the video world.  For instance, last year I wrote about a startup that was using AI in an editing capacity.

The software analyzes footage of video or music to create short movies very much in keeping with the best sports videos on the web.  The output is generally two minutes long and is designed to be watchable (and shareable).

Now, a team from MIT’s renowned Computer Science and Artificial Intelligence Lab (CSAIL), have gone a step further with their own effort that doesn’t even require raw footage to work from.

AI directors

The deep learning algorithm developed by the team has watched some 2 million videos.  As opposed to many machine learning approaches, the videos weren’t labeled, so the algorithm had little idea as to what was going on in each clip.

The algorithm is capable of creating a fresh video from scratch based upon the behaviors of people in the 2 million videos it had previously consumed.  To test out the process, the team then created a 2nd machine learning based algorithm to see if it could spot the AI generated video from the real thing.

Interestingly, they also showed videos to humans on mechanical turk, and whilst most of the time people were able to distinguish between the real and the artificial, the machine was still capable of convincing 20% of viewers that their clip was real.

Now, caveat time.  To date, the video produced by the algorithm can scarcely be called a video, being as it is just 32 frames long, which works out at just over a second.  But, this in itself is still quite an achievement as it’s a significant improvement on what has gone before.  So whilst there is clearly an incredibly long way to go before this kind of technology is capable of doing anything really useful, this is nonetheless an important milestone in that journey.

It suggests a future whereby AI can both create new footage itself, or perhaps more realistically augment existing footage automatically.  For instance, it might be able to create a video out of a still image.

For the moment however, it’s perhaps more likely to serve as a further reminder of the progress being made by AI, and perhaps even assist other researchers who are working on unsupervised machine learning.  The fact that the algorithm was capable of producing reasonably cogent clips from unlabeled objects suggests real progress was made.

Check out the video below to hear more about the project from the research team.