Around a century ago when film stocks and photographs were first coming to light, they faced a number of challenges in capturing the essence of an image.
In addition to the black and white limitation, photography and film methods also struggled to capture other various elements of the color spectrum, rendering many images of famous figures appearing differently than they may have actually looked.
Now, a new AI imaging technique uses color to restyle old photographs in a way that could almost pass for modern day photographs. This colorization method mitigates the main obstacles of cameras and lenses from the olden days – namely, the orthochromatic nature of those tools, meaning that the photo capture device in question incorporated all detected light into the image without discrimination.
The inclusion of all of this light resulted in photos that appeared grainy and noisy, leading to renowned figures such as U.S. president Abraham Lincoln looking far older and wrinklier than he actually was.
These days, especially with the aid of computer graphics, more advanced photographic techniques have taken advantage of the fact that light tends to penetrate the surface of human skin and illuminate the flesh from underneath. This illumination helps to eliminate extra noise and wrinkle marks that marred many images from the early 1900s.
In the past few years, a technique known as Time-Travel Rephotography has helped enhance the quality of older photos by both adding color as well as referencing photos taken by modern day digital cameras in order to ensure realistic turnout for the appearance of human skin.
Created by a team of researchers at Google Research, UC Berkeley and the University of Washington, this technique began by using an archive of contemporary digital portraits to generate sibling photos that shared many traits with the colorized black and white photos.
Moreover, Time-Travel Rephotography also works by identifying the shortcoming in quality characteristic of older black and white photos – such as graininess and noise – and correct these issues for the colorized sibling photo modeled after the original.
A primary advantage to this imaging technique lies in the ability to reflect on how historical figures may have actually appeared in real life. Still, the research teams note that with enough rendering and editing in the wilds of the Internet, the corrected images of these older photographs may end up looking quite different from their original versions.
In fact, as AI imagine enhancing software progresses further all the time, the need for accurate alteration disclaimers grows ever more essential.
The researchers from Microsoft AI team has now created a new algorithm based on artificial intelligence that can restore old images that have suffered from severe degradation. The algorithm uses a deep learning approach to restore old photos.
Conventional restorations can be solved via supervised learning, however, the degradation in real pictures is complex. Moreover, the domain gap among synthetic photos and real old images makes the network fail to generalize. We cannot effectively restore old photos through supervised learning, and the new technique developed by the Microsoft Research team can restore old photos efficiently.
The new AI-based algorithm proposes a novel triplet domain translation network. The new technique leverages real images with massive synthetic photo pairs. Particularly, the Microsoft Research team trains two VAEs (variational autoencoders) that help them to respectively transform old images as well as clean pictures into two latent spaces.
Then, the translation among these latent spaces is learned with synthetic paired information. This translation then generalizes well to real images as the domain gap is now closed in this compact latent space. If there are multiple degradations mixed in one old picture, the research team created a global branch with a partial nonlocal block to address this issue. The partial nonlocal block target the structured defects like dust spots on a photo or scratches.
While the local branch targets the unstructured defects in a photo like blurriness and noises. Then, both of these branches are fused in the latent space. This ultimately leads to enhanced capability of restoring old images from various defects. The new technique outperforms state-of-the-art techniques for restoring old photos.
With the new technique, the image quality is enhanced and it works better than the already existing techniques for restoring old images. In terms of visual quality, this method can prove to be very useful. However, Microsoft has not yet made a demo website available to the public for trying the new technology. We expect that the company will soon make it available publicly to try out the new method for restoring old photos.
More information: Time-Travel Rephotography, arXiv:2012.12261 [cs.CV] arxiv.org/abs/2012.12261
