Olympus E-P1 Imatest Results

Olympus E-P1 Imatest Results

We routinely use Norman Koren's excellent "Imatest" analysis program for quantitative, thoroughly objective analysis of digicam test images. I highly recommend it to our technically-oriented readers, as it's far and away the best, most comprehensive analysis program I've found to date.

My comments below are just brief observations of what we see in the Imatest results. A full discussion of all the data Imatest produces is really beyond the scope of this review: Visit the Imatest web site for a full discussion of what the program measures, how it performs its computations, and how to interpret its output.

Here's some of the results produced by Imatest for the Olympus E-P1:

The Olympus E-P1 showed good color accuracy, with only minor oversaturation of bright reds, deep blues and some purples, dark greens and browns. The E-P1 actually undersaturates some yellows and cyans. Hue accuracy was also generally good, with the typical cyan to blue shift we see in most cameras we test. Average saturation was 105.6% (oversaturated by 5.6%). Average "delta-C" color error was 4.85 after correction for saturation, which is very good, close to the best cameras we've tested. (Delta-C is the same as the more commonly referred to delta-E, but delta-C takes into account only color differences, ignoring luminance variation.) All in all, a very good color response for this class of camera. Mouse over the links below the illustration above to compare results with other recent Four Thirds sensor cameras.

As is true of most SLRs, when using the Adobe RGB color space (which provides a much wider gamut, or range of colors that can be expressed), the Olympus E-P1 delivers more highly saturated color, though the increase compared to sRGB is not as stark as with most SLRs we test (although it's the highest of the group compared above). Average saturation was 114.2%, and average saturation-corrected hue error was 3.99 "delta-C" units, actually slightly more accurate than the sRGB result. Again, mouse over the links below the illustration above to compare results with recent entry-level SLRs. (Sorry, we didn't take an Adobe RGB shot at ISO 200, the E-P1's native ISO.)

This image shows how the Olympus E-P1 actually rendered the colors of the MacBeth chart, compared to a numerically ideal treatment. In each color swatch, the outer perimeter shows the color as actually captured by the camera, the inner square shows the numerically correct color after correcting for the luminance of the photographed chart (as determined by a second-order curve fit to the values of the gray swatches), and the small rectangle inside the inner square shows the numerically correct color, without the luminance correction. This image shows the very good hue and saturation accuracy.

There's a lot in this particular graph, a lot more than we really have room to go into here. (This set of plots has also changed a few revisions back in Imatest. Some of the plots that were once shown here are now shown in other Imatest output. Since we largely focus on the Noise Spectrum plot, we only show the graphic above, which includes that plot.)

In comparing these graphs with those from competing cameras, we've found that the Noise Spectrum graph at lower right is the most important. Cameras that manage to shift their noise spectrum to higher frequencies have much finer-grained noise structures, making their noise less visually objectionable. In the graph above, this would show up as a noise spectrum curve that remained higher on the right side, representing higher noise frequencies.

The Olympus E-P1 does a pretty good job of keeping a lot of the luminance noise energy (indicated by the black line) up at high frequencies. What little low-ISO luminance image noise that's there is fairly fine-grained as a result. Chroma noise at low spatial frequencies is slightly higher, especially in the red channel, but this is not unusual, and the amount of chroma noise in the E-P1's low-ISO images is on par with other cameras in its price class.

Looking at the lower-left plot in the illustration above, the E-P1's overall noise levels are on generally on par with the Panasonic G1 and GH1 in the deepest shadows: Slightly higher than those of the G1 in the deepest shadows, but with considerably less chroma component. On par with the G1 and better than the GH1 in lighter shadows (log density = -1), and better than both in the midtones and highlights. Overall, a very good performance.

Above is the same set of noise data at ISO 200, the Olympus E-P1's native or "base" ISO. As you can see from the graphs, noise is a bit higher than at ISO 100, especially red channel noise at lower frequencies, but still pretty low overall.

Above is the same set of noise data at ISO 1,600. Here, the Noise Spectrum graph is shifted quite a bit toward the left-hand, lower-frequency side than it was at ISO 100, coarsening the "grain" of the image noise patterns quite a bit. The red and blue channels still exhibit a bit more noise at the low end of the frequency spectrum, but track luminance noise very closely at all but the lowest frequencies. Looking at the lower left graph again, noise is higher than both the G1 and GH1 in the deepest shadows, better than the G1 and on par with the GH1 at all higher brightnesses.

Here's the same set of noise data at ISO 3,200. Here again, the Noise Spectrum graph is shifted even more toward the left-hand side, coarsening the "grain" of the image noise patterns further. The tracking of overall noise levels relative to the G1 and GH1 is very similar to what it was at ISO 1,600.

Here's the same set of noise data at ISO 6,400. The Noise Spectrum graph is similar, but notice the noise levels are much higher. (Particularly evident in the pixel noise plot in the upper right corner of the illustration.)

This chart compares the Olympus E-P1's noise performance over a range of ISOs against that of other recent Four-Thirds sensor models. While we continue to show noise plots of this sort because readers ask for them, we each time point out that the noise magnitude is only a small part of the story, the grain pattern being much more important. Here, we can see that the Olympus E-P1's luminance noise magnitude starts out a little higher than average relative to its "competition", and stays higher than the rest up to ISO 800, where it's about equal. At ISOs 1,600 and 3,200, luminance noise levels are lower than its competition, an indication more aggressive noise reduction is at work. The E-P1 is the only camera in this group to offer ISO 6,400, and although it doesn't look too noisy from this graph, the amount of detail left after noise reduction is performed is quite poor, though. Still, really excellent performance overall, when you consider the size of the E-P1. Do keep in mind these measurements are taken with each camera set at default settings, so the shape or position of the curve could be influenced by the settings you choose to use. The Olympus E-P1 offers four levels of noise reduction, with the above data collected using the default, "Standard" setting.

Olympus E-P1 Dynamic Range Analysis

A key parameter in a digital camera is its Dynamic Range, the range of brightness that can be faithfully recorded. At the upper end of the tonal scale, dynamic range is dictated by the point at which the RGB data "saturates" at values of 255, 255, 255. At the lower end of the tonal scale, dynamic range is determined by the point at which there ceases to be any useful difference between adjacent tonal steps. Note the use of the qualifier "useful" in there: While it's tempting to evaluate dynamic range as the maximum number of tonal steps that can be discerned at all, that measure of dynamic range has very little relevance to real-world photography. What we care about as photographers is how much detail we can pull out of the shadows before image noise becomes too objectionable. This, of course, is a very subjective matter, and will vary with the application and even the subject matter in question. (Noise will be much more visible in subjects with large areas of flat tints and subtle shading than it would in subjects with strong, highly contrasting surface texture.)

What makes most sense then, is to specify useful dynamic range in terms of the point at which image noise reaches some agreed-upon threshold. To this end, Imatest computes a number of different dynamic range measurements, based on a variety of image noise thresholds. The noise thresholds are specified in terms of f-stops of equivalent luminance variation in the final image file, and dynamic range is computed for noise thresholds of 1.0 (low image quality), 0.5 (medium image quality), 0.25 (medium-high image quality) and 0.1 (high image quality). For most photographers and most applications, the noise thresholds of 0.5 and 0.25 f-stops are probably the most relevant to the production of acceptable-quality finished images, but many noise-sensitive shooters will insist on the 0.1 f-stop limit for their most critical work.

The image below shows the test results from Imatest for an in-camera JPEG file from the Olympus E-P1 with a nominally-exposed density step target (Stouffer 4110), and the E-P1's settings such as Contrast (0) and Gradation (Normal) at their default positions. The E-P1's native ISO sensitivity of 200 was used, yielding the best dynamic range numbers.

Here, we can see that he tone curve shows pretty good gradation in highlights but shadows trail off much more abruptly. Total dynamic range is good (at 9.78 f-stops), however the score (5.47) at the highest level is poor, likely due to noise in darker tones. Indeed, when deep shadows are closely observed in E-P1 images, a moderate amount of noise is visible, though there is still a lot of subject detail present. We find this far preferable to low noise with detail obliterated by over-aggressive noise reduction, and applaud Olympus on the excellent job they've done, in intelligently managing noise and subject detail.

The illustration above shows the results from Adobe Camera Raw 5.5, with Auto settings (slightly better results are likely possible with manually tweaking, but we weren't able to do better, due to what appeared to be some truncation of the individual color channel tone curves at the highlight end of the tonal range). As can be seen, the score at the highest quality level is much better than the camera JPEG at 8.04 vs 5.47 f-stops, and total dynamic range increased from 9.78 f-stops to 12 f-stops. It bears noting though, that extreme manipulation of RAW files to extend dynamic range can introduce severe color shifts in the resulting files: This dramatically increased dynamic range may only be really usable when converting the images to black and white. (As is the case with most cameras.) With the E-P1, the results from Adobe Camera Raw clipped the green channel slightly in the brightest wedges of the step chart. Also worth noting is that ACR's default noise reduction settings reduced the pixel noise (see the plot in the lower left-hand corner) quite a bit relative to the levels in the in-camera JPEG, which tends to shift scores towards the high quality category.

Dynamic Range, the bottom line:

The net result was that the Olympus E-P1's JPEGs scored rather poorly in Imatest's dynamic range analysis when compared against other current Four-Thirds models, and of course also lags behind most DSLRs with APS-C size sensors. Dynamic range scores from ACR converted RAW files however did quite well, beating some APS-C cameras, but only after extreme manipulation which resulted in false colors at the highlight end of our step target.

To get some perspective, here's a summary of the Olympus E-P1's dynamic range performance, and how it compares to other digital SLRs that we also have Imatest dynamic range data for. (Results are arranged in order of decreasing dynamic range at the "High" quality level.):

The results shown in the table are interesting. One of the first things that struck me when I (Dave) initially looked at test data for a wide range of DSLRs, was that here again, purely analytical measurements don't necessarily correlate all that well with actual photographic experience. There's no question that the Fuji S3 Pro deserved its place atop the list, as its unique "SR" technology did indeed deliver a very obvious improvement in tonal range in the highlight portion of the tonal scale relative to competing models of its day. (Amazing that it's now surpassed by even consumer-level models using today's technology.) I was also surprised to see the analytical results place the original Olympus E-300 as highly as they did, given that our sense of that camera's images was that they were in fact noisier than those of many other DSLRs that we looked at. In the other direction, I was quite surprised to see the Nikon D2x place as low on the listings as it did, given that we found that camera's shadow detail to be little short of amazing.

One thing that's going on here though, is that we tested each camera at its base ISO setting, which should produce best-case noise levels. This is in fact what many photographers will be most interested in, but it does perhaps place some of the Nikons (like the D40) at a disadvantage, as their lowest ISO setting is 200, as compared to the ISO 100 settings available on most other models.

When it comes to the Olympus E-P1, as noted above, its low scores for camera JPEGs reflect Olympus' decision to leave more image noise in its files, so as to not have to trade away as much subject detail. We're finding that we prefer this approach to cameras that smudge away subtle subject detail just to achieve lower noise levels in areas of flat tint, so encourage readers to look at our actual test images (particularly the OUTB shots, which really challenge a camera's dynamic range), rather than making their decision solely on the basis of the numbers in the chart above.

Olympus E-P1 Resolution Chart Test Results

The chart above shows consolidated results from spatial frequency response measurements in both the horizontal and vertical axes. The "MTF 50" numbers tend to correlate best with visual perceptions of sharpness, so those are what we focus on here. The uncorrected resolution figures are 2,602 line widths per picture height in the horizontal direction (corresponding to the vertically-oriented edge), and 2,541 lines along the vertical axis (corresponding to the horizontally-oriented edge), for a combined average of 2,571 LW/PH. Correcting to a "standardized" sharpening with a one-pixel radius reduced the resolution score by quite a bit, resulting in a lower average of 1,999 LW/PH, indicating that the Olympus E-P1 oversharpens a lot at its default settings. The corrected numbers put the E-P1 pretty much on par with its competitors.

To see what's going on, refer to the plots below, which show the actual edge profiles for both horizontal and vertical edges, in both their original and corrected forms. Here, you can see that a high amount of in-camera sharpening is applied in the horizontal direction (oversharpened by 13.3 %), as well as in the vertical direction (12.6 % oversharpened), explaining why standardized sharpening reduced the MTF 50 numbers so much. You should thus turn the camera's sharpening down for optimal results when sharpening in-camera JPEGs post-exposure in Adobe Photoshop or other image editing software. (And, as always, careful processing of RAW files in a good third-party RAW conversion program will likely yield a good bit more fine detail than can be seen in the in-camera JPEGs, regardless of in-camera sharpening settings.)