Canon T1i Imatest Results

Canon T1i Imatest Results

We routinely use Norman Koren's excellent "Imatest" analysis program for quantitative, thoroughly objective analysis of digicam test images. I highly recommend it to our technically-oriented readers, as it's far and away the best, most comprehensive analysis program I've found to date.

My comments below are just brief observations of what we see in the Imatest results. A full discussion of all the data Imatest produces is really beyond the scope of this review: Visit the Imatest web site for a full discussion of what the program measures, how it performs its computations, and how to interpret its output.

Here's some of the results produced by Imatest for the Canon T1i:

The Canon T1i showed pretty good color accuracy overall. Hue accuracy was really excellent (among the very best we've seen), with what little hue shift occurred taking place in the cyans, sky blues, and reds. Average saturation was slightly high at 111.8.3% (oversaturated by 11.8%, mostly in the blues, oranges and dark greens), but not unusually so for a consumer model. Average "delta-C" color error was an very low 3.82 after correction for saturation, which is excellent. All in all, a very good color response for an SLR. Mouse over the links below the illustration above to compare results with competing models.

Using the Adobe RGB color space (which provides a much wider gamut, or range of colors that can be expressed), the Canon T1i delivers more highly saturated color, as is usually the case. The average saturation was 118.7% and average saturation-corrected hue error of 5.31 "delta-C" units, which in this case is slightly less accurate than most of the competition here. Again, mouse over the links below the illustration above to compare results with competing models.

This image shows how the Canon T1i actually rendered the colors of the MacBeth chart, compared to a numerically ideal treatment. In each color swatch, the outer perimeter shows the color as actually captured by the camera, the inner square shows the numerically correct color after correcting for the luminance of the photographed chart (as determined by a second-order curve fit to the values of the gray swatches), and the small rectangle inside the inner square shows the numerically correct color, without the luminance correction. This image shows the Canon T1i's excellent hue accuracy, as well as a gamma curve that results in some overexposure of highly saturated swatches. (This seems to be a fairly common tactic, to produce "bright" color that's appealing to consumers, without further overdoing the saturation.)

There's a lot in this particular graph, a lot more than we have room to go into here. (This set of plots has also changed a little in the more recent versions of Imatest. Some of the plots that were shown here previously are now shown in other Imatest output. Since we largely focus on the Noise Spectrum plot, we'll only show the graphic above, which includes that plot.)

In comparing these graphs with those from competing cameras, I've found that the Noise Spectrum graph at lower right is the most important. Cameras that manage to shift their noise spectrum to higher frequencies have much finer-grained noise structures, making their noise less visually objectionable. In the graph above, this would show up as a noise spectrum curve that remained higher on the right side, representing higher noise frequencies. The champion at this was (and still is) the Canon EOS-1Ds Mark II, which produced remarkably fine-grained image noise, even at very high ISOs.

Above, we see the results at ISO 100, which is the Canon T1i's base ISO. The luminance curve is reasonably flat except for the red, blue and green color channels, which exhibit much higher noise values at lower frequencies. While the overall noise levels are very low, the dominance of low frequencies in the color channels point to some low-level blotchiness.

Above is the same set of noise data at ISO 3,200. Here, we can see the overall noise energy has gone up somewhat (the area under the curves is larger), and the Noise Spectrum graph is still shifted toward the left-hand, lower-frequency side. This time, the color channels more closely matched than at ISO 100, an effect of the Canon T1i's default high ISO noise reduction. The spike in chroma noise at low frequency is limited to the very lowest frequencies, rather than the broader spike we saw at ISO 100: This corresponds to the soft, undulating, cloud-like character of the T1i's chroma noise at high ISOs. (It's laudably subtle, but when you look closely at large areas of flat tint shot with the T1i at ISO 3,200, you can see color noise with a soft, rounded appearance to it, with about 30-40 pixels between peaks.) Looking at the curve in the lower left, the total noise is higher than some competitors in the deepest shadows (far left side of the curve), but pretty competitive otherwise.

Here's the same set of noise data at ISO 6,400. Very similar to ISO 3,200, but overall noise levels are about doubled.

Here's the same set of noise data at ISO 1,2800, the Canon?T1i's maximum ISO. Very similar to ISO 6,400, but even higher noise levels overall. There's also a large shift to the left, indicating a coarser grain pattern, and the low-frequency spike is more pronounced. (What doesn't show in these graphs is how badly low-contrast resolution suffers, especially for red and blue-colored objects.)

This chart compares the Canon T1i's noise performance over a range of ISOs against that of other competing cameras, using default settings. While we continue to show noise plots of this sort because readers ask for them, we each time point out that the noise magnitude is only a small part of the story, the grain pattern being much more important. In the case of the Canon T1i, the noise magnitude starts out slightly higher than most of the competition at ISO 100, and remains higher than all the competition except for the Sony A350 at ISO 800 and the Canon XSi (barely) at ISO 1,600. Keep in mind these are at default settings, so the shape and positions of the curves will be influenced by your settings.

Canon T1i Dynamic Range Analysis

A key parameter in a digital camera is its Dynamic Range, the range of brightness that can be faithfully recorded. At the upper end of the tonal scale, dynamic range is dictated by the point at which the RGB data "saturates" at values of 255, 255, 255. At the lower end of the tonal scale, dynamic range is determined by the point at which there ceases to be any useful difference between adjacent tonal steps. Note the use of the qualifier "useful" in there: While it's tempting to evaluate dynamic range as the maximum number of tonal steps that can be discerned at all, that measure of dynamic range has very little relevance to real-world photography. What we care about as photographers is how much detail we can pull out of the shadows before image noise becomes too objectionable. This, of course, is a very subjective matter, and will vary with the application and even the subject matter in question. (Noise will be much more visible in subjects with large areas of flat tints and subtle shading than it would in subjects with strong, highly contrasting surface texture.)

What makes most sense then, is to specify useful dynamic range in terms of the point at which image noise reaches some agreed-upon threshold. To this end, Imatest computes a number of different dynamic range measurements, based on a variety of image noise thresholds. The noise thresholds are specified in terms of f-stops of equivalent luminance variation in the final image file, and dynamic range is computed for noise thresholds of 1.0 (low image quality), 0.5 (medium image quality), 0.25 (medium-high image quality) and 0.1 (high image quality). For most photographers and most applications, the noise thresholds of 0.5 and 0.25 f-stops are probably the most relevant to the production of acceptable-quality finished images, but many noise-sensitive shooters will insist on the 0.1 f-stop limit for their most critical work.

The image below shows the test results from Imatest for an in-camera JPEG file from the Canon T1i with a nominally-exposed density step target (Stouffer 4110), and the Canon T1i's settings at their default positions, where Auto Lighting Optimization is set to Standard.

These are good results for in-camera JPEGs, though a notch below the top performers when comparing numbers for the High quality threshold (7.43 f-stops). The tone curve shows excellent gradation in highlights but tails-off more abruptly in the shadow end. This can be seen when closely inspecting shots captured by the Canon T1i, where detail is held very nicely in strong highlights, but somewhat less so in the deepest shadows.

The results with Auto Lighting Optimization set to Off are a bit higher at lower quality levels, but identical at the highest level (7.43 f-stops). The total dynamic range figure (irrespective of noise levels) has increased slightly, from 9.96 to 10.3 f-stops.

The Auto Lighting Optimization Low setting had very similar results to the Standard setting.

The Auto Lighting Optimization Strong setting also had very similar results to the other ALO settings, with the same high quality score of 7.43 f-stops but slightly higher overall dynamic range of 10.3 f-stops versus 9.96 f-stops at the default setting.

Processing the T1is RAW (CR2) files through Adobe Camera Raw (ACR) version 5.4 (beta) improved dynamic range by about 1/3 f-stop at the best quality level, and almost 2 stops overall. ACR's automatic settings were used., tweaking the sliders manually may result in slightly better results, but in our own fiddling with curves, we didn't manage to obtain any significant improvement. The response curve here has a more ideal "S" shape to it, where the shadow end of the curve trails off much more gradually than in-camera JPEGs. (Although the highlight side ends a bit more abruptly than we'd prefer.) Worth noting here is that ACR's default noise reduction settings also reduced the pixel noise relative to the levels in the in-camera JPEG. Also important to note is that the degree of manipulation performed by ACR here would likely result in color errors in strong highlights in real-world photographs. (The best performance you're likely to see in a real-world environment will likely be somewhere between the in-camera results and those from Adobe Camera Raw.)

Dynamic Range, the bottom line:

The net result was that the Canon T1i JPEG files came in about average compared to other DSLR models we've tested recently at the highest quality level, and RAW results were a little below average, perhaps hampered by the smaller than average pixel size compared to most current competitors.

To get some perspective, here's a summary of the Canon T1i's dynamic range performance, and how it compares to other digital SLRs that we also have Imatest dynamic range data for. (Results are arranged in order of decreasing dynamic range at the "High" quality level.) Overall, dynamic range for the Canon T1i's in-camera JPEGs is about 2/3 stop below the best consumer-level digital SLRs, while its results from RAW files came up a stop or more low:

The results shown in the table are interesting. One of the first things that struck me when I initially looked at test data for a wide range of d-SLRs, was that here again, purely analytical measurements don't necessarily correlate all that well with actual photographic experience. There's no question that the Fuji S3 Pro deserves its place atop the list, as its unique "SR" technology does indeed deliver a very obvious improvement in tonal range in the highlight portion of the tonal scale. I was surprised to see the analytical results place the Olympus E-300 as highly as they did, given that our sense of that camera's images was that they were in fact noisier than those of many other d-SLRs that we looked at. In the other direction, I was quite surprised to see the Nikon D2x place as low on the listings as it did, given that we found that camera's shadow detail to be little short of amazing.

One thing to note here, though, is that we tested each camera at its lowest (base) ISO setting, which should produce best-case noise levels. This is in fact what many photographers will be most interested in, but it does perhaps place some of the Nikons (like the D40 and D5000) at a disadvantage, as their base ISO setting is 200, as compared to the ISO 100 settings available on most other models.

Canon T1i Resolution Chart Test Results

The chart above shows consolidated results from spatial frequency response measurements in both the horizontal and vertical axes. The "MTF 50" numbers tend to correlate best with visual perceptions of sharpness, so those are what I focus on here. The uncorrected resolution figures are 2,183 line widths per picture height in the horizontal direction (corresponding to the vertically-oriented edge), and 2,207 lines along the vertical axis (corresponding to the horizontally-oriented edge), for a combined average of 2,195 LW/PH. Correcting to a "standardized" sharpening with a one-pixel radius increased both vertical and horizontal resolution significantly, resulting in an average of 2,163 LW/PH, a little lower than the uncorrected value, indicating the default in-camera sharpening is a bit on the aggressive side.

To see what's going on, refer to the plots below, which show the actual edge profiles for both horizontal and vertical edges, in both their original and corrected forms. Here, you can see that the Canon T1i's default in-camera sharpening is fairly strong (a large bump at the top ends). Imatest reports that the horizontal direction (vertical edge) is "oversharpened" by 3.88% while the vertical direction (horizontal edge) is "undersharpened" by only 1.93%. These numbers are pretty close to ideal, which is probably a good default for the intended target market for the Canon T1i. (That said, you should be able to extract more fine detail if you begin with a RAW file, rather than a JPEG.)
.