It is the researchers of Duke University have made the AI tool that can turn fuzzy, unrecognizable images of people’s faces into eerily convincing portraits generated by computers, in more detail than ever. Previous methods can scale up to eight times its original resolution for a face image. But the Duke team has come up with a way to take a handful of pixels and create realistic-looking faces with up to 64 times the size, ‘imagining’ features like fine lines, eyelashes, and stubbles that weren’t there in the first place.
“Never before have super-resolution images been created with this much detail at this resolution,” Duke computer scientist Cynthia Rudin, who led the team, said.
The program cannot be used to recognize individuals; the researchers say: it does not transform an unrecognizable out-of-focus photo from a surveillance camera into a crystal-clear image of a real individual. Instead, it is able to create new faces that do not exist but look plausibly real.
The researchers have focused on faces as the proof of concept, the same technique could, in theory, take low-res shots of almost anything and create sharp, realistic-looking images, with applications ranging from medicine and microscopy to astronomy and satellite imaging, said co-author Sachit Menon ’20, who just graduated from Duke with a double-major in mathematics and computer science.
At the 2020 Conference on Computer Vision and Pattern Recognition (CVPR), held practically from June 14 to June 19, the researchers will present their method, called PULSE.
Traditional approaches take a low-resolution image and ‘guess’ what extra pixels are needed by trying to match them on average in high-resolution images that the computer has seen before. As a result of this average, textured hair and skin areas that may not perfectly align from one pixel to the next end up looking fuzzy and indistinguishable.
Unit Duke has come up with a new strategy. Instead of taking a low-resolution image and slowly adding new detail, the system scores AI-generated examples of high-resolution faces, searching for ones that look like the input image as much as possible when shrunk to the same size.
The team has used a machine learning tool called a “generative adversarial network,” or GAN, which is two neural networks trained on the same photo data set. One network comes up with human faces created by AI which mimics those on which it was trained, while the other takes this output and decides whether it is sufficiently convincing to be mistaken for the real thing.
Rudin said that PULSE can create realistic-looking images from noisy, poor-quality inputs other methods can not. It can spit out any number of uncannily lifelike possibilities from a single blurred image of a face, each of which appears subtly as a different person.
In a few seconds, the system can convert a face image of 16×16-pixels to 1024 x 1024 pixels, adding more than a million pixels, similar to HD resolution. In the computer-generated versions, details such as pores, wrinkles, and wisps of hair that are imperceptible in the low-res photos become crisp and clear.
The researchers asked 40 people to rate 1,440 images generated on a scale of one to five via PULSE and five other scaling methods, and PULSE did the best, scoring nearly as high-quality photos of actual people.
