Olympus E-PL1 Imatest Results

Olympus E-PL1 Imatest Results

We routinely use Norman Koren's excellent "Imatest" analysis program for quantitative, thoroughly objective analysis of digicam test images. I highly recommend it to our technically-oriented readers, as it's far and away the best, most comprehensive analysis program I've found to date.

Our comments below are just brief observations of what we see in the Imatest results. A full discussion of all the data Imatest produces is really beyond the scope of this review: Visit the Imatest web site for a full discussion of what the program measures, how it performs its computations, and how to interpret its output.

Here's some of the results produced by Imatest for the Olympus E-PL1:

The Olympus E-PL1 showed very good overall color accuracy, as well as accurate saturation levels. Hue accuracy was very good, with small hue shifts in reds, oranges, yellows and some greens and purples. The larger shifts in aqua and cyan are very common among the cameras we test; we think they're an effort to produce sky colors that look better on monitors and in print. Average "delta-C" color error was only 4.73 after correction for saturation, which is excellent. (Delta-C is the same as the more commonly referred to delta-E, but delta-C takes into account only color differences, ignoring luminance variation.) Average saturation was 103.4% (oversaturated by only 3.4%), the most accurate of the group, with only minor oversaturation of bright reds, deep blues and some dark greens and browns. The E-PL1 undersaturates some yellows, greens, aquas, cyans and purples. All in all, a very accurate color response for this class of camera. Mouse over the links below the illustration above to compare results with other recent Four Thirds sensor cameras.

As is true of most SLDs and SLRs, when using the Adobe RGB color space (which provides a much wider gamut, or range of colors that can be expressed), the Olympus E-PL1 delivers more highly saturated color. Average saturation was much higher at 113.6%, but average saturation-corrected hue error was 4.03 "delta-C" units, actually slightly more accurate than the sRGB result. Again, mouse over the links below the illustration above to compare results with other Micro Four Thirds SLDs.

This image shows how the Olympus E-PL1 actually rendered the colors of the MacBeth chart, compared to a numerically ideal treatment. In each color swatch, the outer perimeter shows the color as actually captured by the camera, the inner square shows the numerically correct color after correcting for the luminance of the photographed chart (as determined by a second-order curve fit to the values of the gray swatches), and the small rectangle inside the inner square shows the numerically correct color, without the luminance correction. This image shows the generally excellent hue and saturation accuracy, however you can see a slight greenish tinge in the bright yellow and orange swatches, something you can also see in the yellow yarn in our Still Life images. The tinge is very slight, though -- less noticeable than a similar shift in current Panasonic SLD models.

The bottom row of the chart shows exaggerated white balance errors, revealing that the camera produced a slightly cool color balance when using Custom white balance, with a WhiBal card as the reference.

There's a lot in this particular graph, a lot more than we really have room to go into here. (This set of plots has also changed a few revisions back in Imatest. Some of the plots that were once shown here are now shown in other Imatest output. Since we largely focus on the Noise Spectrum plot, we only show the graphic above, which includes that plot.)

In comparing these graphs with those from competing cameras, we've found that the Noise Spectrum graph at lower right is the most important. Cameras that manage to shift their noise spectrum to higher frequencies have much finer-grained noise structures, making their noise less visually objectionable. In the graph above, this would show up as a noise spectrum curve that remained higher on the right side, representing higher noise frequencies.

The Olympus E-PL1 does a very good job of keeping a lot of the luminance noise energy (indicated by the black line) up at high frequencies, performing somewhat better than the E-P2 in this respect. What little low-ISO luminance image noise that's there is fairly fine-grained as a result. Chroma noise at low spatial frequencies is slightly higher than luminance noise, especially in the red and blue channels, but this is not unusual, and the amount of low frequency chroma noise in the E-PL1's low-ISO images is better than Panasonic's Micro Four Thirds cameras. The Panasonic GF1 has red and blue channels that exhibit noise power in the 2 to 2.5 range at the low frequency end.

Looking at the lower-left plot in the illustration above, the E-PL1's component noise levels are nicely matched, with RGB channels following the luminance channel (plotted in gray) fairly closely. Compared to the Panasonic GF1, its closest current competitor, the E-PL1's noise is a bit higher in the deepest shadows. However in the lighter shadows, midtones and highlights, the E-PL1 exhibits lower RMS noise than the GF1. Overall, a very good performance for a Four-Thirds sensor.

Above is the same set of noise data at ISO 200, the Olympus E-PL1's native or "base" ISO. As you can see from the graphs, noise is a bit higher than at ISO 100, especially blue and red channel noise at lower frequencies, but still relatively low.

Above is the same set of noise data at ISO 1,600. Here, the Noise Spectrum graph is shifted quite a bit toward the left-hand, lower-frequency side than it was at ISOs 100 and 200, coarsening the "grain" of the image noise patterns quite a bit. The red and blue channels still exhibit a bit more noise at the low end of the frequency spectrum, but track luminance noise very closely at all but the lowest frequencies. Looking at the lower left graph again, RMS noise is a bit lower than the E-P2 across the board. Compared to the GF1, noise is slightly higher in the deepest shadows, but lower than the Panasonic at higher brightnesses.

Here's the same set of noise data at ISO 3,200. Here again, the Noise Spectrum graph is shifted even more toward the left-hand side, coarsening the "grain" of the image noise patterns further. The tracking of overall noise levels relative to the GF1 is similar to what it was at ISO 1,600, with the E-PL1 having a slight edge at all brightness levels except for the darkest shadows and brightest highlights. The E-PL1 also edges out the E-P2 across all brightnesses

This chart compares the Olympus E-PL1's noise performance over a range of ISOs against that of other recent Micro Four-Thirds models. While we continue to show noise plots of this sort because readers ask for them, we each time point out that the noise magnitude is only a small part of the story, the grain pattern being much more important. Here, we can see that the Olympus E-PL1's luminance noise magnitude graph shows significantly lower noise levels than the E-P1 and E-P2, and is in fact, lower than all the competition at most ISOs as well. Inspecting the E-PL1's JPEGs closely, it can be seen that they are indeed cleaner than the E-P1/P2, however it is also evident that default noise reduction is stronger. There is little difference in noise levels when you compare E-P2 and E-PL1 RAW files, though the E-PL1's converted files do look slightly cleaner. This is an excellent performance overall for its class, and you can always adjust the noise reduction settings to suite your taste. The Olympus E-PL1 offers four levels of noise reduction, with the above data collected using the default, "Standard" setting.

Olympus E-PL1 Dynamic Range Analysis

A key parameter in a digital camera is its Dynamic Range, the range of brightness that can be faithfully recorded. At the upper end of the tonal scale, dynamic range is dictated by the point at which the RGB data "saturates" at values of 255, 255, 255. At the lower end of the tonal scale, dynamic range is determined by the point at which there ceases to be any useful difference between adjacent tonal steps. Note the use of the qualifier "useful" in there: While it's tempting to evaluate dynamic range as the maximum number of tonal steps that can be discerned at all, that measure of dynamic range has very little relevance to real-world photography. What we care about as photographers is how much detail we can pull out of the shadows before image noise becomes too objectionable. This, of course, is a very subjective matter, and will vary with the application and even the subject matter in question. (Noise will be much more visible in subjects with large areas of flat tints and subtle shading than it would in subjects with strong, highly contrasting surface texture.)

What makes most sense then, is to specify useful dynamic range in terms of the point at which image noise reaches some agreed-upon threshold. To this end, Imatest computes a number of different dynamic range measurements, based on a variety of image noise thresholds. The noise thresholds are specified in terms of f-stops of equivalent luminance variation in the final image file, and dynamic range is computed for noise thresholds of 1.0 (low image quality), 0.5 (medium image quality), 0.25 (medium-high image quality) and 0.1 (high image quality). For most photographers and most applications, the noise thresholds of 0.5 and 0.25 f-stops are probably the most relevant to the production of acceptable-quality finished images, but many noise-sensitive shooters will insist on the 0.1 f-stop limit for their most critical work.

The images below show the test results from Imatest for an in-camera JPEG file from the Olympus E-PL1 with a nominally-exposed density step target (Stouffer 4110), and the E-PL1's settings such as Contrast (0) and Gradation (Normal) at their default positions.

Here, we can see that he tone curve shows pretty nice gradation in the highlights, but the shadow end doesn't trail off as gradually. Total dynamic range was good at 10.2 f-stops, however the score at the highest level was 5.89 f-stops. That score is pretty low, perhaps a result of the nice-looking tone curve, which does cause the camera to use some of its available output dynamic range to produce the pretty tail at the highlight end of the curve. There's also moderate amounts of noise in the deep shadows, though there is still a lot of subject detail present. In general, we find this far preferable to low noise with detail obliterated by over-aggressive noise reduction, and applaud Olympus on the excellent job they've done in their recent models, in intelligently managing noise and subject detail.

The illustrations above show the results from Adobe Camera Raw 5.7, with Auto settings (slightly better results are likely possible with manually tweaking, but we weren't able to do better, due to what appeared to be some truncation of the individual color channel data at the highlight end of the tonal range). As can be seen, the score at the highest quality level at ISO 200 is much better than the camera JPEG at 7.39 vs 5.89 f-stops, and total dynamic range increased from 10.2 f-stops to 11.5 f-stops. It bears noting though, that extreme manipulation of RAW files to extend dynamic range can introduce severe color shifts in the resulting files: This dramatically increased dynamic range may only be really usable when converting the images to black and white. (As is the case with most cameras.) With the E-PL1, the results from Adobe Camera Raw clipped the green channel slightly in the brightest wedges of the step chart. Also worth noting is that ACR's default noise reduction settings reduced the pixel noise (see the plot in the lower left-hand corner) a bit relative to the levels in the in-camera JPEG, which tends to shift scores towards the high quality category.

Dynamic Range, the bottom line:

The Olympus E-PL1's JPEG Imatest dynamic range scores were similar to the E-P2's, just slightly better. However, they both scored rather poorly when compared against other current Four-Thirds models, and of course lag behind most DSLRs with APS-C size sensors. Dynamic range scores from ACR converted E-PL1 RAW files did much better than the JPEGs, but still scored a bit below average.

To get some perspective, here's a summary of the Olympus E-PL1's dynamic range performance, and how it compares to other digital SLRs that we also have Imatest dynamic range data for. (Results are arranged in order of decreasing dynamic range at the "High" quality level.):

Not that this test is repeatable to within 1/3 EV according to the Imatest website, so differences of less than 0.33 can be ignored.

When it comes to the Olympus E-PL1, as noted above, its low scores for camera JPEGs reflect Olympus' decision to leave more image noise in its files, so as to not have to trade away as much subject detail. We're finding that we prefer this approach to cameras that smudge away subtle subject detail just to achieve lower noise levels in areas of flat tint, so we encourage readers to look at our actual test images (particularly the OUTB and FAR shots, which really challenge a camera's dynamic range), rather than making their decision solely on the basis of the numbers in the chart above.

Olympus E-PL1 Resolution Chart Test Results

The chart above shows consolidated results from spatial frequency response measurements in both the horizontal and vertical axes. The "MTF 50" numbers tend to correlate best with visual perceptions of sharpness, so those are what we focus on here. The uncorrected resolution figures are 2,208 line widths per picture height in the horizontal direction (corresponding to the vertically-oriented edge), and 2,119 lines along the vertical axis (corresponding to the horizontally-oriented edge), for a combined average of 2,163 LW/PH, a little lower than the E-P2's uncorrected number of 2,572. Correcting to a "standardized" sharpening with a one-pixel radius reduced the resolution score by a bit, resulting in a slightly lower average of 2,111 LW/PH (a little higher than the E-P2's 2,002 result), indicating that the Olympus E-PL1 only slightly oversharpens at its default settings. The numbers put the E-PL1 pretty much on par with its competitors.

To see what's going on, refer to the plots below, which show the actual edge profiles for both horizontal and vertical edges, in both their original and corrected forms. Here, you can see that an almost ideal amount of in-camera sharpening is applied in the horizontal direction (oversharpened by only 2.22 %), as well as in the vertical direction (1.16 % oversharpened), explaining why standardized sharpening didn't change the MTF 50 numbers by much.

As always, though, careful processing of RAW files in a good third-party RAW conversion program will yield a good bit more fine detail than can be seen in the in-camera JPEGs, regardless of in-camera sharpening.

Note: We don't feature SFR-based LW/PH resolution numbers more prominently in our reviews (eg, outside the Imatest pages) because we've found that they're *very* sensitive to minor differences in in-camera image processing. Relatively small changes in the amount of in-camera sharpening can have a large effect on the resulting resolution numbers. Imatest attempts to compensate for this by adjusting to a "standard" sharpening, but this approach can't completely undo what happens inside the cameras, and so often gives inconsistent results. Sometimes the "standardized" sharpening happens to just match the shape of the edge profile with the in-camera sharpening applied, and you'll get wildly high results. At other times, it will tend to correct in the opposite direction. Unfortunately, ignoring the in-camera sharpening entirely can result in even greater discrepancies, particularly between models from different manufacturers. Turning off sharpening in the camera may or may not fully eliminate the sharpening, so simply turning off sharpening in the camera JPEGs isn't a reliable solution. It also wouldn't be the way most people shoot the cameras. We could process RAW files with no sharpening, but then that'd only suit the people working primarily or exclusively from RAW, and would open another can of worms as to what RAW converter was used, etc, etc.

The bottom line is that numbers for resolution only take you so far. Detail handling and edge acuity are very complex issues; ones that don't easily boil down to a single number. The best approach is to simply look look at the broad array of standardized test shots we take with each camera, to the point of downloading and printing them with whatever processing you'd use if you owned the camera and shot with it. See how the differences stack up for you visually, and make your decision on that, rather than on abstract resolution numbers.