Adobe Brings its Awesome Content-Aware Fill to Video

I still remember the first time I used content-aware fill in Photoshop back in 2010. The idea that you could just draw a line around the thing you wanted to remove from an image, and the software could just make it disappear, realistically recreating the background behind it blew my tiny mind.

Now, the latest version of After Effects can intelligently remove things from video footage as well – and as a measure of how far things have come in the last nine years, my first reaction was "huh, I'm surprised that took so long." After all, the software has much more information to work with in a multi-frame video – some frames might reveal what's actually behind the object you want to remove.

Of course, it took so long because it's an exponentially more complex task than working on a still image. As well as tracking multiple moving objects and backgrounds through dozens or even hundreds of frames, the software not only has to paint the unwanted object out in each frame, but it needs to do it in such a way that the motion of the patched area makes sense in context of the rest of the video. It's a sensational piece of programming wizardry.

The new After Effects version doesn't go quite as far as the amazing SceneStitch technology we saw a teaser for back in 2017, but it'll be a huge time saver for video professionals and will open up simple object removal to pretty much anyone who's got a copy of the latest version of After Effects.

In simple terms, all a user needs to do is select an object in a video frame, be it a still or moving object, and track it through the footage to be edited. Once it's tracked, you can then either let After Effects attempt to automatically fix it, or you can throw a few frames out to Photoshop to correct it yourself and have more of a say in the final result.

As with the content-aware tool in Photoshop, the results will vary depending on what's happening in the rest of your frame, but this is by far the easiest object removal tech available on the mass market, and in Adobe's demonstration videos it does a pretty stunning job.

Loz Blain