How Your Phone Became Seeable In The Dark

New York (CNN) Open Instagram at any time and it won’t take long to find a clear photo of the night sky, skyline after dark, or a dimly lit restaurant. Shots like this used to require sophisticated cameras, which can now often be taken from your phone. You already have it in your pocket.

Tech companies like Apple, Samsung, and Google are investing resources to improve. Night photography options with camera features becoming more and more a key selling point for smartphones.

Earlier this month, Google introduced a faster version of Night Sight mode, which uses AI algorithms to lighten and brighten images in dark environments, to more Pixel models. Available on models dating back to iPhone 11, Apple’s Night Mode was touted as the best feature of last year’s iPhone 14 lineup thanks to its improved camera system.

these are Tools have come a long way in the last few years thanks to significant advances in artificial intelligence technology and image processing. Sharper, faster, and more resilient to difficult photography situations. And smartphone makers aren’t done yet.

Lian Jye Su, Artificial Intelligence Analyst at ABI Research, said: “[This] It just powers smartphone companies with AI-enhanced image and video processing. ”

Silicon Valley’s new AI arms race over chatbots has received a lot of attention lately, but pushing the development of more sophisticated AI tools will further improve nighttime photography, allowing smartphones to work even in the dark. may become visible.

Usage

Samsung’s Night Mode feature is available on various Galaxy models, but is optimized for the premium S23 Ultra smartphone. We promise to do what was unthinkable just 5-10 years ago. It’s all about enabling your phone to take sharp pictures in low light.

This feature is designed to minimize what is called “noise”. In photography, it usually refers to poor lighting conditions, long exposure times, and other factors that can compromise image quality.

The secret to reducing noise, according to the company, is the combination of the S23 Ultra’s adaptive 200M pixel sensor.After the shutter button is pressed, Samsung uses advanced multi-frame processing to Merge multiple images into one photo and let AI automatically adjust the photo as needed.

Joshua Cho, executive vice president of visual solutions team at Samsung, said: “The Galaxy S23 Ultra instantly detects which details should be preserved and which noises should be removed.”

For Samsung and other tech companies, AI algorithms are Essential for delivering photos taken in the dark. “The AI ​​training process is based on a large number of images that have been adjusted and annotated by experts, and the AI ​​learns parameters to adjust every photo taken in low light,” Su said. I will explain.

For example, algorithms identify the appropriate exposure level, determine the correct color palette and gradation under certain lighting conditions, artificially sharpen blurry faces and objects, and modify them. However, the final result is The person taking the photos saw it in real time, but some might argue it was a technical trick of the hand.

Google is also working hard to reduce noise in photos. Its AI-powered Nightscape mode feature captures bursts of long-exposure frames. Then use something called HDR+ Bracketing to create multiple photos with different settings. After taking the photos, it combines those images to create “sharper pictures” that are “incredibly bright and detailed” even in low-light environments, said Alex Schiffhauer, his manager of Group Products at Google. increase.

Effective, but can introduce a small but noticeable delay before the image is ready. But according to Schiffhauer, Google plans to speed up this process even more in upcoming iterations of his Pixel. “We want a world where customers can get Night Sight quality without having to sit still for a few seconds,” Schiffhauer said.

Google also has an astrophotography feature that allows you to take pictures of the night sky without having to tweak exposure or other settings. According to the company, the algorithm detects details in the sky and emphasizes them to make them stand out.

Apple has long been rumored to be working on an astrophotography feature, but some iPhone 14 Pro Max users have managed to capture sky photos using the existing Night Mode tool. When your device detects a low-light environment, Night Mode turns on to capture more detail and brighten your shots. (The company did not respond to requests for details on how the algorithm works.)

AI can change the image, Final result According to Gartner analyst Bill Ray, each of these features also depends on the lens of the phone. Traditional camera lenses sit a few centimeters from the sensor, but the limited space in mobile phones often means you have to cram things in, resulting in a shallow depth of field, especially in dark environments. , the image quality may be degraded.

“Lens quality is still important, and how the phone copes with the lack of depth,” Ray said.

next big thing

Night photography on mobile phones has come a long way, but a hot new technology could take it even further.

Generative AI, technology Powering the viral chatbot ChatGPT . But these AI systems, trained on vast amounts of online data, also have the potential to edit and process images.

“In recent years, generative AI models have also been used for photo-editing functions such as background removal and replacement,” Su said. Su said that if the technology is added to smartphone photography systems, night mode could eventually become even more powerful.

Big tech companies, including Google, have already fully incorporated this technology into other parts of their business. in the meantime, Smartphone chipset vendors such as Qualcomm and MediaTek are looking to support more generative AI applications natively in consumer devices, Su said. These include image and video enhancements.

“But we’re still about two to three years away from having a limited version of this on smartphones,” he said.