Canon 5D Mark IV Dual Pixel RAW

Canon 5D IV Dual Pixel RAW Insights

Is it a post-capture focus adjustment or not?

by Dave Etchells | Posted: 08/25/2016

The Canon 5D Mark IV introduces a new technology called Dual Pixel Raw for the first time. What it amounts to is the ability to shift the apparent focus very slightly after a picture has been taken. Alternately -- or in combination with the focus tweak, if you prefer -- you can make small changes in the bokeh or out of focus areas of the image, or you can reduce ghosting to a degree.

Canon was very careful not to use the word "focus" when describing Dual Pixel Raw to us, saying that they don't want people to get the idea that they can make significant post-focus adjustments in software. Instead, they refer to it as a "micro adjustment". That obviously begs the question of what's being adjusted, and as long as you promise not to tell anyone, we'll go ahead and say that it's focus. (Or bokeh, or ghosting reduction, as noted above.) But do see the full discussion below, to understand why Canon is being careful not to use the "f-word" ;-)

A very small adjustment is possible after tripping the shutter

The amount of the post-capture (sort-of-focus) adjustment is indeed very small, though; think in terms of fractions of an inch at a shooting distance of several feet. The application that most immediately came to mind would be a portrait shot captured with a very wide-aperture lens, where the plane of focus was just barely off. Think in terms of a photo where the model's eyelashes were in sharp focus, but the catchlight in her eye was soft; that's what we're talking about. (The trick to having a portrait look sharp when the DOF is well under an inch is to make sure the catchlights in your subject's eyes are sharp. If the catchlights are crisp, people looking at the picture will feel that it's a "sharp photo".)

The little video clip above gives you some idea of the magnitude of the effect. (Macro shots like this one are another area where the small amount of adjustment might prove useful.) The clip is a screenshot of a beta version of Canon's DPP software, showing the extent of the +/- adjustment available.

The trade-off: Hefty file sizes

Dual Pixel Raw does come at a price, though; the raw files saved in this mode are twice the size of normal, weighing in at a hefty ~72 MB each, and as far as we know can only be processed by an updated version of Digital Photo Professional. (It's possible that third-party raw converters may be able to read out the data for each frame separately, but we doubt they'll be able to execute the effect unless Canon licenses the technology to them. Adobe Camera Raw and Lightroom? We'll see, but we'd say don't hold your breath.

How does the 5D Mark IV's Dual Pixel Raw feature work?

The new Raw mode builds upon Canon's excellent Dual-Pixel CMOS AF technology. This advanced on-sensor phase-detection autofocus system has been a mainstay of its higher-end DSLRs ever since the EOS 70D was introduced in July of 2013. The split-pixel design means that every pixel on the image sensor has two halves, one looking left and the other looking right. This in turn means that every pixel can serve as a phase-detection pixel, potentially usable for autofocus.

We don't need to go into the workings of phase-detection AF (or PDAF for short) very deeply for this article; Wikipedia has a good overview, and this article has a clever Flash animation which helps you see what goes on in a DSLR as you change the focus.

The basic idea of PDAF is pretty straightforward. When an image is in perfect focus, light rays coming from the right and left sides of the lens meet perfectly at the surface of the image sensor. If the lens is out of focus in one direction, the rays meet and then cross each other in front of the sensor, and if it's out of focus the other way, the rays end up meeting behind the sensor (or would, if the sensor wasn't there.)

Phase-detection autofocus works by looking at the light arriving from each side of the lens separately, comparing the light pattern falling on groups of phase-detect pixels on either side of a centerline. The AF system basically slides the separate sets of pixel data coming from each side of the lens past each other, and looks for how much of a shift is required for the light patterns to match exactly. That shift tells the camera how much the lens needs to be adjusted to achieve focus, and in what direction.

All this is by way of background, to explain what Dual Pixel Raw is all about, and what we think it's doing. The Canon reps we've spoken to as of this writing didn't have deep technical details, but we can speculate a bit about what's going on.

How Dual Pixel Raw works: A speculative answer

Given how phase-detection AF works, and that essentially every pixel on the 5D IV's sensor has two halves, looking in opposite directions from each other, it makes sense that the point of focus could be shifted slightly, with some fancy image processing.

As noted above, if some part of the subject is out of focus, the light patterns falling on the sensor from the left and right sides of the lens will be offset a bit from each other. Normally that's the end of the story, but what if we went back and tried to undo that offset in the image data after the fact? Normal AF processing can determine the amount of offset between the left-looking and right-looking views, so why not actually shift the image data by that amount, to bring the two sides into alignment as they should be? Wouldn't that result in a sharper image?

The answer is yes, at least up to a point. Likewise, we could throw something further out of focus by increasing the offset between the two sets of data.

We're pretty sure that this is what the Canon 5D Mark IV is doing in its Dual Pixel Raw exposure mode. Rather than combining data from the two halves of each pixel before saving the image to the memory card, it saves two separate images, one using just the right-looking half-pixels, the other using just the left-looking ones.

Later, the two sets of data can be analyzed in the same way the AF system would, to find how much offset there is between the two overlaid half-images for each point in the subject. The point of apparent best focus can be changed slightly by shifting the two sets of pixel data relative to each other before recombining them into single, full pixels. Shift the right and left-looking images in one direction before combining them and you'll have changed the apparent plane of focus to be closer to the camera. Shift the half-images in the other direction, and the apparent plane of focus will be further away.

Obviously, there are limits to this. When part of a subject is out of focus, more happens than just the two sets of pixel data shifting relative to each other. The image is blurring radially, not just left to right and up and down, so simple X-Y shifts can't fully compensate for the loss of focus. The whole point spread function is changing, and it's a complex two-dimensional relationship that varies non-linearly depending on the amount and direction of the focus error.

For small adjustments, though, simply shifting the right- and left-looking images relative to each other before combining them can make a noticeable difference. It's only enough to make very small shifts in the apparent focal distance, but as noted above, it can be enough to sharpen the catchlights in a model's eyes or crispen-up important detail in a macro shot.

(Considering the above, it's clear why Canon is being so careful not to use the word "focus" or "focus adjustment" in describing Dual Pixel Raw. It's not remotely a true focus shift, but can do a convincing job of looking like one for small adjustments.)

We'll experiment more with this feature once we get a production sample of the 5D Mark IV and a matching copy of Canon's Digital Photo Professional raw converter to do the processing with. It seems like it could be useful in some situations, but do note that the maximum amount of apparent focus shift you can obtain is quite small, and it comes at the expense of raw files twice the normal size. (They're around ~72 MB at base ISO, vs the already-large 36 MB normal ones). Given how small the maximum adjustment is, it may not be worth the size penalty, except in specific situations. But of course when you need it for critical purposes, it really seems a great tool.

Stay tuned, we'll have more once we've had a chance to experiment with production hardware and software. Meanwhile, kudos to Canon for a clever secondary use of its Dual-Pixel CMOS AF technology!

Wait, there's more: Dual Pixel RAW can provide more headroom in highlights!

Our good friend Iliah Borg -- he and Alex Tutubalin are the developers of the superb FastRawViewer and RawDigger apps and the related LibRaw library -- has pointed out what strikes us as an even more exciting aspect of the technology: Properly processing a Dual Pixel RAW exposure can provide you with additional highlight detail, potentially allowing you to salvage a shot that was accidentally overexposed.

See Iliah's article for how he came to the realization that Dual Pixel RAW could be repurposed to work in this manner!