Sony IMX378: Comprehensive Breakdown of the Google Pixel’s Sensor and its Features

IMX378 Overview

We reached out to Sony to try to learn a bit more about the IMX378 sensor that is used by the upcoming Google Pixel and Pixel XL phones, as well as by the Xiaomi Mi 5S. Unfortunately, Sony was not able to distribute the datasheet for the Exmor RS IMX378 sensor just yet, but they were extremely helpful, and were able to provide us with some previously unreleased information about the IMX378.

First up, the very name itself was wrong. Despite rumors stating that it would be part of the Exmor R line of Backside Illuminated (BSI) CMOS sensors like the IMX377 before it that was used in the Nexus 5X and Nexus 6P, our contact at Sony has informed us that the IMX378 will instead be considered part of Sony's Exmor RS line of Stacked BSI CMOS sensors.

While many things have remained the same from the IMX377 to the IMX378, including the pixel size (1.55 μm) and sensor size (7.81 mm), there have been a couple key features added. Namely it is now a stacked BSI CMOS design, it has PDAF, it adds Sony's SME-HDR technology, and it has better support for high frame rate (slow motion) video.

Stacked BSI CMOS

Backside illumination by itself is an extremely useful feature that has become almost standard in flagship smartphones for the last few years, starting with the HTC Evo 4G in 2010. It allows the camera to capture substantially more light (at the cost of more noise) by moving some of the structure that traditionally sat in front of the photodiode on front illuminated sensors, behind it.

Surprisingly, unlike most camera technology, backside illumination originally started appearing in phones before DSLRs, thanks in large part due to the difficulties with creating larger BSI sensors. The first BSI APS-C sensor was the Samsung S5KVB2 that was found in their NX1 camera from 2014, and the first full-frame sensor was the Sony Exmor R IMX251 that was found in the Sony α7R II from last year.

Stacked BSI CMOS technology takes this one step further by moving more of the circuitry from the front layer onto the supporting substrate behind the photodiodes. This not only allows Sony to substantially reduce the size of the image sensor (allowing for larger sensors in the same footprint), but also allows Sony to print the pixels and circuits separately (even on different manufacturing processes), reducing the risk of defects, improving yields, and allowing for more specialization between the photodiodes and the supporting circuitry.

PDAF

The IMX378 adds Phase Detection Autofocus, which last year's Nexus phones and the IMX377 did not support. It allows the camera to effectively use the differences in light intensity between different points on the sensor to identify if the object that the camera is trying to focus on is in front of or behind the focus point, and adjust the sensor accordingly. This is a huge improvement both in terms of speed and accuracy over the traditional contrast-based autofocus that we've seen on many cameras in the past. As a result, we've seen an absolute explosion of phones using PDAF, and it has become a huge marketing buzzword which is held up as a centerpiece of camera marketing across the industry.

While not quite as quick to focus as the Dual Photodiode PDAF that the Samsung Galaxy S7 has (also known as "Dual Pixel PDAF" and "Duo Pixel Autofocus"), which allows every single pixel to be used for phase detection by including two photodiodes per pixel, the merger of PDAF and laser autofocus should still be a potent combination.

High Frame Rate

There's been a lot of talk lately about high frame rate cameras (both for consumer applications, and in professional filmmaking). Being able to shoot at higher frame rates can be used both to create incredibly smooth videos at regular speed (which can be fantastic for sports and other high-speed scenarios) and to create some really interesting videos when you slow everything down.

Unfortunately, it is extremely difficult to shoot video at higher frame rates, and even when your camera sensor can shoot at higher frame rates, it can be difficult for the phone's image signal processor to keep up. That is why while the IMX377 used in the Nexus 5X and 6P could shoot 720p video at 300 Hz and 1080p video at 120 Hz, we only saw 120 Hz 720p from the Nexus 5X and 240 Hz 720p from the 6P. The IMX377 was also capable of 60 Hz 4k video, despite the Nexus devices being limited to 30 Hz.

The Pixel phones are both able to bring this up to 120 Hz 1080p video and 240 Hz 720p video thanks in part to improvements related to the IMX378, which sees an increase in capabilities of up to 240 Hz at 1080p.

The sensor is also able to shoot full resolution burst shots faster, stepping up to 60 Hz at 10 bit output and 40 Hz at 12 bit output (up from 40 Hz and 35 Hz respectively), which should help reduce the amount of motion blur and camera shake when using HDR+.

SME-HDR

Traditionally, HDR for video has been a trade-off. You either had to cut the frame rate in half, or you had to cut the resolution in half. As a result, many OEMs haven't even bothered with it, with Samsung and Sony being among the few that do implement it. Even the Samsung Galaxy Note 7 is limited to 1080p 30 Hz recording due in part to the heavy computational cost of HDR video.

The first of the two main traditional methods for HDR video, which Red Digital Cinema Camera Company calls HDRx and which Sony calls Digital Overlap HDR (DOL-HDR), works by taking two consecutive images, one exposed darker and one exposed lighter, and merging them together to create a single video frame. While this allows you to keep the full resolution of the camera (and set different shutter speeds for the two separate frames), it can often result in issues due to the time gap between the two frames (especially with fast moving objects). Additionally, it can be very difficult for the processor to keep up, as with DOL-HDR, the phone's ISP handles merging the separate frames together.

The other traditional method, which Sony calls Binning Multiplexed Exposure HDR (BME-HDR), sets a different exposure setting for every pair of two lines of pixels in the sensor to create two half resolution images at the same time, which are then merged together into one HDR frame for the video. While this method avoids the issues associated with HDRx, namely a reduction in frame rate, it has other issues, specifically the reduction in resolution and the limits on how the exposure can be changed between the two sets of lines.

Spatially Multiplexed Exposure (SME-HDR) is a new method that Sony is using to allow them to shoot HDR at the full resolution and at the full frame rate that the sensor is capable of. It is a variant of Spatially Varying Exposure that uses proprietary algorithms to allow Sony to capture the information from the dark and light pixels, which are arranged in a checkerboard style pattern, and infer the full resolution image for both the dark and light exposure images.

While Sony unfortunately was not able to disclose the exact pattern that they use at this time (and they may never be able to disclose it — companies tend to play their cards very close to their chest when it comes to cutting edge technology, like that which we see in HDR, with even Google having their own proprietary algorithm for HDR photos, known as HDR+), there is still some publicly available information that we can use to piece together how it may be accomplished. A couple of papers have been published by Shree K. Nayar of Columbia University (one of which was in collaboration with Tomoo Mitsunaga of Sony) that contain different ways to use Spatially Varying Exposure, and different layouts that can achieve it. Below is an example of a layout with four levels of exposure on an RGBG image sensor. This layout claims to be able to achieve single capture full resolution HDR images with only around a 20% loss in spatial resolution, depending on the scenario (the same accomplishment that Sony claims for SME-HDR).

Sony has used SME-HDR in a couple image sensors already, including in the IMX214 that has seen a lot of popularity lately (being used in the Asus Zenfone 3 Laser, the Moto Z, and the Xperia X Performance), but is a new addition to the IMX378 compared to the IMX377 that was used last year. It allows the camera sensor to output both 10 bit full resolution and 4k video at 60 Hz with SME-HDR active. While a bottleneck elsewhere in the process will result in a lower limit, this is a fantastic improvement over what the IMX377 was capable of, and is a sign of good things to come in the future.

One of the big improvements of the IMX378 over the IMX377 is that it is able to handle more of the image processing on-chip, reducing the workload of the ISP (although the ISP is still able to request the RAW image data, depending on how the OEM decides to use the sensor). It can handle many small things like defect correction and mirroring locally, but more importantly, it can also handle BME-HDR or SME-HDR without having to involve the ISP. That could potentially be a major difference going forwards by freeing up some overhead for the ISP on future phones.

We would like to thank Sony once again for all the help with creating this article. We really appreciate the lengths that Sony went to in helping ensure the accuracy and depth of this feature, especially in allowing us to uncover some previously-unreleased information about the IMX378.

That being said, it really is a shame that it is so hard to access some of this information, even basic product information. When companies try to put information on their websites, it often can be rather inaccessible and incomplete, in large part because it is often treated as a secondary concern of the company's employees, who are more focused on their main work. One dedicated person handling public relations can make a huge difference in terms of making this type of information available and accessible to the general public, and we're seeing some people trying to do just that in their free time. Even on the Sony Exmor Wikipedia article itself, where over the course of a couple months a single person in their spare time laid most of the foundation to take it from a nearly useless 1,715 byte article that had been mostly the same for years, into the ~50,000 byte article which we see there today with 185 distinct editors. An article that is arguably the best repository of information about the Sony Exmor sensor line available online, and we can see a very similar pattern on other articles. A single dedicated writer can make a substantial difference in how easily customers can compare different products, and in how educated interested consumers are about the subject, which can have far-reaching effects. But that's a topic for another time.

As always, we're left wondering how these hardware changes will affect the devices themselves. We quite clearly will not be getting 4k 60 Hz HDR video (and may not be getting HDR video at all, as Google has not mentioned it yet), but the faster full resolution shooting likely will help substantially with HDR+, and we will see the improvements of the newer sensor trickle into the phone in other similar small but substantial ways as well.

While DXOMark lists the Pixel phones as performing slightly better than the Samsung Galaxy S7 and HTC 10, many of the things that gave the Pixel phones that small lead were major software improvements like HDR+ (which produces absolutely fantastic results, and which DXOMark dedicated an entire section of their review to) and Google's special EIS system (which can work in tandem with OIS) that samples the gyroscope 200 times a second to provide some of the best Electronic Image Stabilization we have ever seen. Yes, the Pixel phones have a great camera, but could they have been even better with OIS and Dual Pixel PDAF added in? Absolutely.

Don't get me wrong, as I said, the Pixel phones have an absolutely stunning camera, but you can't really blame me for wanting more, especially when the path to those improvements is so clear (and when the phones are priced at full flagship pricing, where you expect the best of the best). There's always going to be a part of me that wants more, that wants better battery life, faster processors, better battery life, brighter and more vivid screens, louder speakers, better cameras, more storage, better battery life, and most importantly, better battery life (again). That being said, the Pixel phones have many small fantastic features that could come together to create a truly promising device, which I am excited to see.

from xda-developers http://ift.tt/2e7rYFD
via IFTTT

Moto and Updates: More Planned Obsolesce from Lenovo’s Motorola

Planned obsolescence is a tricky grey area in every field. Advancements in science and technology allow equipment and components to last longer and perform reliably, but these improvements also have a very direct short-term impact on the sales of your future devices.

After all, if the previous device is still working, that is one less compelling reason for a consumer to purchase the next device that the company will release a short while later. A lot of decisions that smartphone makers take also play directly within the scope of planned obsolescence, even if the prime motive behind the decision was something else. Sealed batteries, forced updates that slow down the device and as is the case with Android OEMs, not providing updates at all — everyone is guilty of factoring in some element of obsolescence in their releases.

Just to take our minds off phones with sealed batteries for a change, let's talk about Lenovo and Motorola. Before Lenovo acquired Motorola, the company was praised for its Moto X lineup, how they performed for the money they asked for, and how quickly the devices got updated owing to their minimal skin over pure Android. The Moto X 2013, on Verizon of all carriers, received its Android 4.4 KitKat update just around three weeks after KitKat was announced — even before the Nexus 4 got the update!

The Moto X from 2013 was a Motorola phone that received an update faster than a Nexus

But ever since Lenovo acquired Motorola, things have been…different. And we're not talking only in the terms of the delay in receiving updates to an OS skin which is a fairly minimal touch-up to stock Android. We're also talking about some of the weird decisions regarding updates that the company has taken in the recent past. To recall, Lenovo had just about killed off the update support for the Moto E 2015 only 6 months after the device's release. But after a lot of bad press, the company yielded and promised an Android 6.0 update for the device in some regions.

We're at yet another of these perplexing update decisions from Lenovo with regards to Motorola branded devices. Motorola's recent Android 7.0 Nougat announcement for its devices included a long list of devices, which included devices like the Moto G4 Play and the Moto X Play. Curiously, as brought to our attention by our forum members, the list is missing devices like the Moto G3 and the Moto G Turbo Edition.

The Moto G3 was launched in July 2015 while the Moto G4 Play was launched in May 2016. Ironically, the G4 Play sports the same CPU and GPU as the G3 but at a lower clock speed (go figure!). As mentioned above, the device is getting the official Android 7.0 Nougat update while the G3 will not. Similar, yet slightly more befuddling, situation exists with the Moto G Turbo Edition which was launched in November 2015 and shares the exact same CPU and GPU as the Moto X Play which was launched earlier in July 2015. The Moto X Play will receive Nougat, while the newer Moto G Turbo Edition will not.

Now factor in this: Android 6.0 was officially released in October 2015, and Android 7.0 was officially released in August 2016. The oldest devices from the above mentioned four are the Moto G3 and the Moto X Play, both of which were released in July 2015, but only one of them is receiving its Android 7.0 update. A device released a few months later with the same SoC as the update-receiving phone, is however, not getting the update. Confused? We are too.

There is no clear update policy at play here with Motorola and Lenovo. One cannot argue that they are favoring only the most recent of devices, because with the Moto X Play in the picture, they are not.

One also cannot argue that the devices in question sport weak and outdated SoCs which makes the update technically infeasible, because other devices with the same SoC released by Motorola itself, are being promised an update. Yes, SoC is not the only roadblock in bringing an update to a device. But with the number of hardware similarities between the Moto G3 and the Moto G4 Play, a large part of the update work is already done in some form by Motorola.

One could also come on to argue that pricing of the devices is what Motorola used to decide which devices are worth updating. But if you take a look at the predecessors of the Moto G3, both the Moto G and the Moto G2 received two Android updates that spanned over version numbers — KitKat and Lollipop for the Moto G, and Lollipop and Marshmallow for the Moto G2. The Moto G3 will sadly, remain on Marshmallow under Motorola's current plan.

With the MotoMaker options, the Moto G 2015 provided great hardware. But sadly, poor software update support.

So that brings us back to the starting argument of this article — planned obsolescence. Motorola's update plans are nothing short of planned obsolescence. By not providing the latest in terms of software, Motorola and Lenovo hope to fleece customers into purchasing newer phones that are the same in terms of SoC. The lack of update is the mechanism put into place to induce consumers to abandon perfectly capable and working hardware, in favor of newer devices that are more of sidegrades than upgrades. Motorola is not even supporting devices in its budget lineup for 18 months of software updates anymore, as is evident from the Moto G3.

What you do get now from Motorola and Lenovo is uncertainty. You simply do not know if the device you purchase will be updated for another year or two. Granted, the low end of the market is not where one should expect the most in terms of after-sales updates, but Motorola's recent actions reaffirm that they are back with the rest of the Android OEMs into ignoring the budget devices. Which is pretty ironic, considering that the Moto G, the budget hero, was Motorola's best selling smartphone of all time. And to be fair, while we keep using the name "Motorola" here, we really should be calling out Lenovo — these issues and other controversies became prominent after Lenovo's acquisition, as it was under Google that Motorola has begun releasing updates faster.

Our forum members are petitioning for the Moto G3 and the Moto G Turbo Edition to receive some Nougat love from Motorola. If you would like to support their cause, please visit the forum thread. We hope bringing light to this issue manages to convince Lenovo once more that there's value in software updates.

What are your thoughts on Motorola's recent update plans? Are you looking forward to purchasing a Motorola device in the future? Let us know your thoughts in the comments below!

from xda-developers http://ift.tt/2d7pKKp
via IFTTT

Petition for Getting Android 7.0 Official Update for Moto G 2015

Our forum members have launched a petition to get Lenovo to include the Moto G 2015 in its official update list, owing to similarities in hardware with other devices in that list. Help spread the word!

from xda-developers http://ift.tt/2e1krZI
via IFTTT

Samsung Begins Mass-Producing the Exynos 7 Dual 7270 SoC for Wearables

Samsung has just announced mass production of their first wearable SoC that is built upon the company's 14nm FinFET process technology. The company has been expanding their 14nm technology since it was introduced back in 2015, and now they are bringing it to all sorts of wearables. This also marks the first in its class to offer full connectivity and LTE modem integration right into the chip.

The Exynos 7 Dual 7270 is made up of 2 ARM Cortex-A53 cores, which has been able to cut down on power usage by 20% when compared to its predecessor built on a 28nm process. The SoC also features an integrated Cat.4 LTE 2CA modem so the wearable doesn't have to be connected to your smartphone in order to use cellular service. On top of all that, the Exynos 7 Dual 7270 also includes the traditional Wi-Fi, Bluetooth, FM and GNSS features.

The chip also utilizes Samsung's SiP(system-in-package)-ePoP (embedded package-on-package) technology. This means the Exynos 7 Dual 7270 stacks the AP, DRAM, NAND flash memory and the power management IC so that it can reduce the height by 30% compared to its predecessors. This, along with the 14nm FinFet process technology makes it ideal for wearable technology like smartwatches.

Samsung has said the Exynos 7 Dual 7270 is currently in mass production and that a a reference platform that consists of the chip, NFC and various other sensors are available for OEMs who want to utilize the technology. This could end up being good competition to Qualcomm's Snapdragon Wear 2100 chip, and it's something that the company needs since it just released a revised earnings guidance for the third quarter of 2016.

Samsung needs to do anything they can to make up the financial lose that will come from the current Galaxy Note 7 battery debacle.

Source: Samsung Newsroom

from xda-developers http://ift.tt/2dclbt9
via IFTTT

Android 7.1 Developer Preview to Begin in October

Google has finally announced details of the Android 7.1 Developer Preview program and tells us that it will be available on supported Nexus devices later this month. Just like with the Android N Developer Preview, anyone with a supported device will be able to enroll via the Android Beta program. If you already have a device enrolled in the Android Beta program then you won't have to change anything.

Google tells us that the Nexus 5X, Nexus 6P and the Pixel C will be the first to try out the Android 7.1 Developer Preview. We aren't told if it will be done in phases like the Android N Developer Preview was, but they do say that additional details about the program will be revealed soon. Currently, Google has the final version of Android 7.1.x scheduled to hit the Nexus 5X, Nexus 6P, Pixel C, Nexus Player and supported Android One devices in "early December."

With Android 7.1, Google is introducing API level 25 and says the update delivers productivity, security, and performance improvements along with optimizations and bug fixes over the current 7.0 version. This update introduces the app shortcuts API, circular app icon resources, enhanced live wallpaper metadata, image keyboard support, and the storage manager intent. On the carrier side of things, 7.1 also includes new APIs for multi-endpoint calling and new telephony configuration options.

As with all developer previews that Google has done for Android, the whole idea is to get the update into the hands of developers so they prepare their apps for the update. Google says they've already been working with OEMs in hopes to get them pushing out an Android 7.1 OTA update as quickly as possible.

Source: Android Developers Blog

from xda-developers http://ift.tt/2dJhKy0
via IFTTT

Signs of Life For Android Pay in Canada: Interac Imagery Added in Version 1.7

There has been a lot of background activity recently surrounding Android Pay's global rollout, and an additional sign of the work that Google is putting in to prepare for their Canadian launch has just surfaced in Android Pay version 1.7. Android Pay now contains the first stages of Interac support.

The additions to the Android Pay app are relatively small, just the Interac logo and one line of code ("<string name="tp_interac">Interac</string>"), but they are a sign of what is going on behind the scenes. Google is in negotiations with the local banks (and with Interac), and has been rumoured to be working on bringing Android Pay to Canada some time this year (although that is a deadline that may end up being missed, much like how Samsung Pay and Apple Pay ran into difficulties in Canada). It will be exciting to see what happens when Android Pay reaches Canada, especially with Canada's high adoption rate for tap to pay debit and credit cards (which are compatible with Android Pay).

For those that aren't familiar with the Canadian banking environment, Interac is a non-profit joint venture between the major Canadian banks, which handles the standardization of interbank transactions in Canada, especially for Debit cards. There are over 59,000 ATMs, 83 banks, and 450,000 stores that use Interac in Canada, making it the de facto standard.

Story Via: 9To5 Google

from xda-developers http://ift.tt/2e9dUwC
via IFTTT

N-ify Xposed Module Updated to Enable Google Assistant on Marshmallow

The popular Xposed module N-ify which adds Nougat features to older Android versions has been updated and now users of the 6.6.14.21.arm64 Google app can access Google Assistant even on Marshmallow.

from xda-developers http://ift.tt/2dOyoMb
via IFTTT