Sony NEX-3 Imatest Results

Sony NEX-3 Imatest Results

We routinely use Norman Koren's excellent "Imatest" analysis program for quantitative, thoroughly objective analysis of digicam test images. I highly recommend it to our technically-oriented readers, as it's far and away the best, most comprehensive analysis program I've found to date.

My comments below are just brief observations of what we see in the Imatest results. A full discussion of all the data Imatest produces is really beyond the scope of this review: Visit the Imatest web site for a full discussion of what the program measures, how it performs its computations, and how to interpret its output.

Here's some of the results produced by Imatest for the Sony NEX-3:

The Sony NEX-3 showed generally good color accuracy, with only minor oversaturation of bright reds and deep blues, and to a lesser extent, some oranges and greens. Hue accuracy was also generally good, with the typical (typical meaning significant) cyan shift we see in most cameras we test, and also a little shift in some reds and oranges. Average saturation for the Sony NEX-3 was 111.1% (oversaturated by 11.1%). Average "delta-C" color error was 4.97 after correction for saturation, which puts it in the top tier. (Delta-C is the same as the more commonly referred to delta-E, but delta-C takes into account only color differences, ignoring luminance variation.) All in all, a very good color response for this class of camera. Mouse over the links below the illustration above to compare results with other recent consumer SLDs.

Most SLRs/SLDs dramatically oversaturate colors when shooting in Adobe RGB mode, and the Sony NEX-3 follows suit. Average saturation was 116.3%, and average saturation-corrected hue error was 5.26 "delta-C" units, just slightly less accurate than the sRGB result. Again, mouse over the links below the illustration above to compare results with recent SLDs.

This image shows how the Sony NEX-3 actually rendered the colors of the MacBeth chart, compared to a numerically ideal treatment. In each color swatch, the outer perimeter shows the color as actually captured by the camera, the inner square shows the numerically correct color after correcting for the luminance of the photographed chart (as determined by a second-order curve fit to the values of the gray swatches), and the small rectangle inside the inner square shows the numerically correct color, without the luminance correction. This image shows the generally very good hue and saturation accuracy. Most colors are rendered reasonably close to the right brightness, the most obvious exception being the cyan and light blue swatches, which are noticeably brighter than the luminance-corrected versions. You can also see a slight greenish tinge in the orange swatch on the right side of the figure. Overall, though, the NEX-3's color is quite good: A little brighter than that of the Olympus and Panasonic SLDs (both of which are less oversaturated), and without the orange through yellow hue shifts of the Panasonic line.

 

As always, there's more in this particular graph than we really have room to go into here. (Also note that this set of plots has also changed a few revisions back in Imatest. Some of the plots that were once shown here are now shown in other Imatest output. Since we largely focus on the Noise Spectrum plot, we only show the graphic above, which includes that plot.)

In comparing these graphs with those from competing cameras, we've found that the Noise Spectrum graph at lower right is often the most important. Cameras that manage to shift their noise spectrum to higher frequencies have much finer-grained noise structures, making their noise less visually objectionable. In the graph above, this would show up as a noise spectrum curve that remained higher on the right side, representing higher noise frequencies.

In the case of the Sony NEX-3, we find an interesting situation at its base ISO of 200: The luminance noise is unusually fine-grained, with very little rise in the noise spectrum at low frequencies. On the other hand, the color channels (and especially the red one) have a lot of energy lumped up on the low-frequency side of the curve. The bottom line on the NEX-3 is that there's very little noise at all, regardless of the frequency distribution, so the big, puffy clouds of color aren't visible unless you really do some really radical tonal adjustments.

Above is the same set of noise data at ISO 1,600. Here, the overall Noise Spectrum graph is shifted a fair bit toward the left-hand, lower-frequency side than it was at ISO 200, coarsening the "grain" of the image noise patterns, but there's much less difference between the color channels in this plot, with green tracking the luminance noise very closely, and only the red and blue-channel noise being a bit elevated at the lowest frequencies. In the past, it's been unusual to find a greater difference between chroma and luminance noise at lower ISOs than at higher ones, although it's becoming more common. Overall noise levels here are lower than those of competing SLD cameras, at least as of this writing in early July, 2010, and are very much on par with the best SLRs using APS-C size sensors.

Here's the same set of noise data at ISO 3,200. Here again, the Noise Spectrum graph is shifted even more toward the left-hand side, further coarsening the "grain" of the image noise patterns further. Overall noise levels are again excellent, though, the Sony NEX-3 very much holding its own against pretty much all comers. (Short of full-frame camera models, which are obviously in a very different category.)

This chart compares the Sony NEX-3's noise performance over a range of ISOs against that of other SLR and SLD cameras capable of video recording. While we continue to show noise plots of this sort because readers ask for them, we each time point out that the noise magnitude is only a small part of the story, the grain pattern being much more important, not to mention what the camera does to achieve a given noise level. (Some cameras obliterate subject detail along with noise, so the camera with the lowest noise levels on MacBeth chart swatches may not necessarily produce the most appealing images.)

All the caveats about absolute noise levels not necessarily being that applicable to real images, the Sony NEX-3 does look very impressive here: Its midtone noise level essentially equals or betters that of all the other cameras shown. As expected, results were very similar to the NEX-5. (The NEX-3's luminance noise is practically identical to the NEX-5 up to ISO 3,200 and a bit higher at ISOs 6,400 and 12,800, but that is likely due to slightly different exposures at the higher ISOs, not a real difference in the sensors.) The results might be easy to dismiss, if it turned out that its detail handling was poor, but our close visual examination of key test images revealed that its detail handling is also among the best we've seen from this size sensor. All in all, a very impressive performance, and a real step forward for Sony.

 

Sony NEX-3 Dynamic Range Analysis

A key parameter in a digital camera is its Dynamic Range, the range of brightness that can be faithfully recorded. At the upper end of the tonal scale, dynamic range is dictated by the point at which the RGB data "saturates" at values of 255, 255, 255. At the lower end of the tonal scale, dynamic range is determined by the point at which there ceases to be any useful difference between adjacent tonal steps. Note the use of the qualifier "useful" in there: While it's tempting to evaluate dynamic range as the maximum number of tonal steps that can be discerned at all, that measure of dynamic range has very little relevance to real-world photography. What we care about as photographers is how much detail we can pull out of the shadows before image noise becomes too objectionable. This, of course, is a very subjective matter, and will vary with the application and even the subject matter in question. (Noise will be much more visible in subjects with large areas of flat tints and subtle shading than it would in subjects with strong, highly contrasting surface texture.)

What makes most sense then, is to specify useful dynamic range in terms of the point at which image noise reaches some agreed-upon threshold. To this end, Imatest computes a number of different dynamic range measurements, based on a variety of image noise thresholds. The noise thresholds are specified in terms of f-stops of equivalent luminance variation in the final image file, and dynamic range is computed for noise thresholds of 1.0 (low image quality), 0.5 (medium image quality), 0.25 (medium-high image quality) and 0.1 (high image quality). For most photographers and most applications, the noise thresholds of 0.5 and 0.25 f-stops are probably the most relevant to the production of acceptable-quality finished images, but many noise-sensitive shooters will insist on the 0.1 f-stop limit for their most critical work.

The image below shows the test results from Imatest for an in-camera JPEG file from the Sony NEX-3 with a nominally-exposed density step target (Stouffer 4110), and its settings such as DRO and HDR turned off.

Here, we can see that the tone curve maintains excellent gradation in both the highlights and shadows, with a very film-like "S". These results handily beat those from all other current SLD models, and in fact are very much on par with those from the best SLRs with APS-C sensors.

Once again, it would be easy to dismiss this excellent performance, if it turned out that Sony's anti-noise processing was wiping out subject detail in the process: But that's not what's happening; the Sony NEX-3 does an excellent job of preserving subtle subject detail, far more so than any Sony camera we've seen to date. (Apart from its sibling the NEX-5, of course, shares the same sensor.)

Processing the Sony NEX-3's RAW (.ARW) files through Adobe Camera Raw (ACR) version 6.2 beta increased dynamic range by more than an f-stop at the highest quality level (8.87 f-stops), compared to the best in-camera JPEG (7.44 f-stops) while total dynamic range reported increased from 10.6 to 11.9 f-stops. These results were obtained by using ACR's automatic settings; slightly better results may be possible by manually tweaking from there. It's worth noting here that ACR's default noise reduction settings reduced overall noise (see the plot in the lower left-hand corner) relative to the levels in the in-camera JPEG, which would tend to boost the dynamic range numbers for the High Quality threshold. Also, the extreme highlight recovery being performed by ACR here would likely produce color errors in strong highlights of natural subjects.

Dynamic Range, the bottom line:

The net result was that the Sony NEX-3 showed excellent dynamic range, performing as well as the NEX-5 (within the margin of error) and easily beating competing SLD camera models we've tested to date (including the Samsung NX10, which also has an APS-C size sensor). What's more, it's really in the upper percentile ranking of all the SLRs we've tested with APS-C sensors. A very impressive step up for Sony.

To get some perspective, here's a summary of the Sony NEX-3's dynamic range performance, and how it compares to other digital SLRs that we also have Imatest dynamic range data for. (Results are arranged in order of decreasing dynamic range at the "High" quality level.):

Note that this test is repeatable to within 1/3 EV according to the Imatest website, so differences of less than 0.33 can be ignored.

As noted above, the Sony NEX-3's JPEG dynamic range at the highest quality level was better than its current SLD competition, with the exception of the NEX-5 (of course), and the Panasonic GH1. - But the GH1 drops out of the race below that top quality level, as you have to go to more than the maximum 1.0 stops of noise before it breaks above 7.77 stops of dynamic range. Comparing the NEX-3 with full-size SLRs with APS-C sensors, we find that it holds its own very well indeed: It's within a few tenths of a stop of all the top performers. (Keep in mind that Imatest only claims repeatability on dynamic range measurements to within 1/3 stop, so the difference compared to the NEX-5 scores is negligible.)

Sony NEX-3 Resolution Chart Test Results

The chart above shows consolidated results from spatial frequency response measurements in both the horizontal and vertical axes. The "MTF 50" numbers tend to correlate best with visual perceptions of sharpness, so those are what we focus on here. The uncorrected resolution figures are 2,580 line widths per picture height in the horizontal direction (corresponding to the vertically-oriented edge), and 2,104 lines along the vertical axis (corresponding to the horizontally-oriented edge), for a combined average of 2,342 LW/PH. Correcting to a "standardized" sharpening with a one-pixel radius increased both horizontal and vertical numbers very slightly, resulting in an average of 2,391 LW/PH.

The Sony NEX-3 delivers good resolution, quite competitive with 14-15-megapixel DSLRs currently on the market. Images straight from the camera show good detail, although we can't yet report on the results obtainable via Adobe Camera Raw, because as of this writing (early July, 2010), it does not support the NEX-3 yet.

To see what's going on, refer to the plots below, which show the actual edge profiles for both horizontal and vertical edges, in both their original and corrected forms. Here, you can see that a moderate amount of in-camera sharpening is applied in both the horizontal and vertical directions, but with a modest amount of overshoot, and fairly small extent (oversharpened by only 0.33% in the horizontal direction, and actually undersharpened by 3.4% in the vertical, according to Imatest's calculations). As usual, you should turn the camera's sharpening down a little for optimal results when sharpening in-camera JPEGs post-exposure in Adobe Photoshop or other image editing software, but the NEX-3's in-camera sharpening is better controlled than we see on many cameras.

Note: We don't feature SFR-based LW/PH resolution numbers more prominently in our reviews (eg, outside the Imatest pages) because we've found that they're *very* sensitive to minor differences in in-camera image processing. Relatively small changes in the amount of in-camera sharpening can have a large effect on the resulting resolution numbers. Imatest attempts to compensate for this by adjusting to a "standard" sharpening, but this approach can't completely undo what happens inside the cameras, and so often gives inconsistent results. Sometimes the "standardized" sharpening happens to just match the shape of the edge profile with the in-camera sharpening applied, and you'll get wildly high results. At other times, it will tend to correct in the opposite direction. Unfortunately, ignoring the in-camera sharpening entirely can result in even greater discrepancies, particularly between models from different manufacturers. Turning off sharpening in the camera may or may not fully eliminate the sharpening, so simply turning off sharpening in the camera JPEGs isn't a reliable solution. It also wouldn't be the way most people shoot the cameras. We could process RAW files with no sharpening (as shown below), but then that'd only suit the people working primarily or exclusively from RAW, and would open another can of worms as to what RAW converter was used, etc, etc.

The bottom line is that numbers for resolution only take you so far. Detail handling and edge acuity are very complex issues; ones that don't easily boil down to a single number. The best approach is to simply look look at the broad array of standardized test shots we take with each camera, to the point of downloading and printing them with whatever processing you'd use if you owned the camera and shot with it. See how the differences stack up for you visually, and make your decision on that, rather than on abstract resolution numbers.