All you need to do to wipe an object from footage is draw a box around it, and the software takes care of the rest for you. Here are a couple examples of the AI in action:
As you will notice, while the algorithm gets rid of the person crossing the street in a rather convincing way, it leaves some traces of foul play. The upper white line bends as the camera moves with the now-gone subject. In other cases, the object removal process leaves smudges in the footage:
The software was built by a developer going by the pseudonym zllrunning, who has since uploaded it to GitHub. To get it off the ground, the developer says he relied on two other training models, called SiamMask and Deep Video Inpainting. The former was used to track moving objects in videos, and the latter to mask them out of the footage. In all fairness, there is other software out there that does the same trick. Indeed, researchers from the Nanyang Technological University in Singapore teased an algorithm that performs the same function back in May 2019. Adobe too showcased a similar feature in its video editing app After Effects earlier in 2019. Here’s what it looks like: Unlike After Effects though, the software above is open source, which means you can browse its code and modify it to fit your needs (if you happen to posses the technical know-how). In the meantime, those who want to try the object removal AI can head to this GitHub repository for a step-by-step guide on how to install it. [H/T HackerNews, SyncedReview]
