Ricoh reveals pixel-shifting high-resolution mode: Coming soon to a camera near you?

Here's a little follow-up from CP+2015 that got misplaced until now, but one with big implications for Pentax fans: Ricoh officially talked about their pixel-shifting super-resolution technology, almost certainly coming in their full-frame DSLR slated for release later this year - And it may not be just new Ricoh/Pentax DSLRs that get to benefit from the tech! There are also some tidbits about other Ricoh tech, including their latest Aero-Bright II lens coating technology that promises to banish flare from reflections off lenses' glass elements.

While Olympus made big news at the CP+ 2015 trade show with the E-M5 II's pixel-shifting super-resolution mode, Ricoh quietly showcased a number of advanced technologies, including a similar pixel-shifting super-res capability. Other than a rumor tweeted by a Ricoh-associated pro photographer prior to the show, it's received relatively little comment on the 'net so far. Perhaps that's because the display talking about it was all in Japanese, tucked into the back of their booth, and a presentation I happened to see being given was also entirely in Japanese. That's too bad, because we think it's a pretty significant technology.

When we first heard of Olympus' approach in the E-M5 II, our first thought was "of course!", followed very quickly by "hey, Ricoh/Pentax could do this, too!" In fact, it's almost surprising that Olympus came out with the feature before Ricoh, given that Ricoh/Pentax was who developed the unique switchable mechanical anti-alias filter simulation capability back with their K-3. Now, our questions have been answered with Ricoh's technology announcement at CP+ 2015.

 

The conventional Bayer color filter array (CFA) design uses a repeating pattern of red, green, and blue color filters, arranged in 2x2 pixel cells. Each cell has one red, one blue, and two green pixels. Cameras using the Bayer CFA pattern have to interpolate image data, to fill-in missing color and luminance information. This results in both lower effective resolution and a tendency towards color aliasing (little flecks of color) whenever there are very abrupt light/dark transitions in the subject.

Unlike Olympus' approach, which grabs 8 frames and uses a half-pixel shift between two groups of four to interpolate additional pixels, Ricoh's method uses just four shots, to bring the four pixels of each Bayer color filter array cell into alignment with each other. The resulting file has the same number of pixels as conventional shots, but each pixel contains full, uninterpolated RGB data. (Reminiscent in that sense of Sigma's Foveon sensors.)

 

Ricoh's special pixel-shift enhancement mode involves shifting the sensor in one-pixel increments up and down and back and forth, so after four exposures, each pixel contains full RGB data. The end image has the same number of megapixels as the sensor itself, but the data is of much higher quality, with no interpolation needed to "fill in" missing color or luminance information.

As with any such pixel-shifting approach, both camera and subject must be motionless, else you get something much worse-looking than ordinary motion blur. But the result is not only sharper-looking images, but the complete elimination of color moiré or interpolation artifacts. You'll still get luminance moiré of course, but cleaning up color aliasing is a big deal; in our experience, color moiré causes visible artifacts much more often than luminance moiré does.

At the show, Ricoh studiously avoided any mention of when this technology might come to market, in what body, etc. Given their touting of it at the same time as they formally announced their forthcoming full-frame body, though, it doesn't take a lot of imagination to put two and two together.

 

Ricoh showed image results with/without the pixel-shift technology, but the differences were honestly a bit hard to see on the monitor they were displayed on. From our past experience with Sigma/Foveon sensors, though, the difference can be pretty noticeable, especially with fine color detail. Foveon's calculations say there's about a 2x increase in effective resolution vs what you'd get with a Bayer-striped array, which seems about right, based on what we've seen.

But wait, there's more! When I asked a Ricoh executive whether this pixel-shifting image enhancement tech might come to any of their existing SLRs via firmware updates, he of course said that he couldn't comment on future developments. He did say, though "But Ricoh believes in taking care of their customers" - and encouraged me to quote him on that. :-)

So will we see pixel-shift resolution technology coming to some existing Pentax SLRs via firmware updates? While far from giving a definitive answer, it seems clear that Ricoh is looking in that direction. Pretty exciting news, I think!

Ricoh's new Aero Bright II lens coating

Ricoh also announced a significant update to their Aero Bright lens-coating technology. Aero Bright is Ricoh's version of nano-particle coating, a technology that's had a big impact on lens flare over the last few years. Manufacturers tend to be pretty tight-lipped about the specifics, but we do know that there are various flavors of nano-structure coatings for camera lenses. They all basically attempt to provide a more gradual transition of refractive index at air/glass boundaries. It's the abrupt change in refractive index that produces reflections; no index change, no reflection, big, abrupt index change, lots of reflection. If you can make the refractive index change gradually instead of abruptly, you can dramatically reduce the reflections.

 

This chart shows graphs of reflectance (vertical axis) vs wavelength (horizontal axis) for four different coating technologies. From top to bottom, they are Ricoh/Pentax's older, conventional coatings, their newer, "HD" coatings, their original Aero Bright coating, and the latest Aero Bright II version. The level of flare suppression with the latest AeroBright II technology is pretty remarkable.

As I said, there are several approaches to doing this (Canon's method in particular seems different than others I've heard about), but I think a lot work the way Ricoh's does: You basically deposit tiny particles of some substance on the glass, with gaps between the particles. (Who knows what the particles are made of, that seems to be a closely-guarded secret, but I guess it would be safe to say that it's something with a refractive index somewhere between those of air and glass.) If the particles are small enough, the mixture of particles and air produces a graded refractive index that cuts the reflections.

 

By increasing the size and number of voids in the Aero Bright II coating, Pentax produced a refractive index that was a better balance between those of air and glass.

The key advancement with the Aero Bright II seems to be that the amount of air space between the nano-particles has been increased significantly, from 40% empty space in Aero Bright I to 58% in Aero Bright II. The end result is that the average refractive index of the Aero Bright II layer has decreased from 1.27 to 1.18. For comparison, air and most gasses have refractive indices close to 1.0, while most glasses fall are in the range of 1.46 to 1.62. Almost all solids and liquids have indices greater than 1.3 (Wikipedia, refractive index) The only things that get lower are aerogels, so a refractive index of 1.18 is pretty incredibly low.

 

Ricoh's advanced nano-coating is just the icing on a layer-cake of conventional anti-reflective coatings. I hadn't realized just now "nano" nanocoating particles were, but the electron micrograph at lower right shows that the particles in Ricoh's Aero Bright II coating are on the order of 20-30 nanometers. For comparison, the wavelength of green light is about 550 nanometers. So these particles are something on the order of 1/20th the wavelengths of visible light!

It turns out that Ricoh (and I'm sure other companies as well) don't rely on the nano-coating to do the whole job by itself. The nano-coating is applied on top of a conventional stack of alternating low- and high refractive index coatings, laid down by vacuum deposition. The illustration above shows the "stack" of optical coatings that make up Aero Bright II. The top layer is the nano-coat, while the light layers below represent high refractive-index coatings, and the darker ones represent low refractive-index ones. (While we expected the bottom layer, next to the glass, to be high-index, we were surprised that the top layer, right below the nano-coating appears to be high-index as well. We'd have thought you'd want to have a low-index one on that side, but maybe the air or nano-coating itself serves that purpose.)

(Traditional lens coatings work by an entirely different approach than nano-coatings; they're basically interference filters, with each coating layer typically knocking out reflections in a given wavelength range. They're lots better than no coating at all, but nano-coatings are in another league altogether.)

 

Ricoh's skin ton rendering technology in the 645z (and we believe in the recently-announced K-S1 and K-S2 as well) handles skintones via a separate color-management process from the camera's general-purpose color mapping.

Super Skin Tones

While it wasn't a new technology (it's apparently in the 645Z, and I believe will be in the forthcoming the K-S1 and K-S2), Ricoh had a display that talked about their technology for accurate rendering of skin tones.

Skin tones are tough, as they're "memory" colors for humans; we really, really know what they're supposed to look like. Sometimes, a camera's color management setup can be such that images are generally properly white-balanced and color-accurate, but skin tones may still look off.

Ricoh's approach in the 645Z is to create a mapping of the color space occupied by skin tones, in terms of lightness, hue, and color saturation. Each data point in this sub-color space is then mapped into the RGB color what would most faithfully reproduce the skin tone it's associated with. Anything that falls within the boundaries of this sub-color space is translated according to the skin-tone mapping function, rather than relying up on the camera's default color management processing.

Ricoh spatial frequency-based "clarity" feature

This was interesting, although I may not have deciphered fully what was being shown here. (Perhaps one of our readers fluent in Kanji could provide a better translation in the comments below?) What I think this is talking about is a "clarity" function to intelligently enhance local contrast.

 

Ricoh's new "clarity" enhancing function employs a stack of synthetic images, each having progressively lower spatial frequency content. Looking across these images, the algorithm is able to determine areas where local contrast enhancement would improve the image.

After an image is captured, it's decomposed into a series of images with lower spatial frequency cutoffs. (Basically progressively stronger low-pass filters, created by the image processor.) These different versions of the image are compared with each other, to identify areas of locally-significant tonal difference. The algorithm can thereby pick out key details for enhancement. This new function will first be found in the upcoming Pentax K-S2.

 

This is a side-by-side comparison of the results with the new clarity-enhancement turned on and off - although it might be a little futile, showing a low-resolution version of a bad screenshot with glaring lights reflecting off the original monitor :-/

What's particularly significant about this development is that my host pointed out it was developed within Ricoh's R&D organization. He cited it as an example of Ricoh's R&D horsepower translating into new capabilities for their camera products. In their previous incarnations as an independent company and then under Hoya's auspices, Pentax never had this sort of engineering depth to call upon.

It'll be interesting to see what other things come out of Ricoh's R&D labs going forward!

[Do we have any technically-oriented Japanese readers out there? I'm not at all sure I got the details right on this last part. Click here for a large version of the original screenshot, if you'd be so kind as to translate it for other readers and ourselves in the comments below.]