Sony Cyber-shot DSC-R1Sony "breaks the mold" with a unique SLR/all-in-one hybrid design.<<Reference: Datasheet :(Previous) | (Next): DSC-R1 Sample Images>> DSC-R1 IMATEST ResultsReview First Posted: 09/20/2005, Updated: 11/18/2005 |
Detailed analysis of the Sony DSC-R1 images, from Imatest(tm)
I've recently begun using Norman Koren's excellent "Imatest" analysis program for quantitative, thoroughly objective analysis of digicam test images. I highly commend it to our technically-oriented readers, as it's far and away the best, most comprehensive analysis program I've found to date.
My comments below are just brief observations of what I see in the Imatest results. A full discussion of all the data Imatest produces is really beyond the scope of this review: Visit the Imatest web site for a full discussion of what the program measures, how it performs its computations, and how to interpret its output.
Here's some of the results produced by Imatest for the Sony DSC-R1:
Color Accuracy
The Sony DSC-R1 showed very good hue accuracy, but tended to oversaturate additive primary colors (reds, blues, and greens) a fair bit. Its images looked bright and vibrant, but reds in particular looked a little hot. Average saturation was 113.5% (oversaturated by 13.5%, mostly in the reds, blues, and greens), average "delta-E" color error was 4.47. (Hue error, after correction for saturation.)
Working in the Adobe RGB color space, most cameras tend to render colors with a bit less saturation, as would befit a color space used primarily by professionals and others more concerned about accurate color rendition. The R1 goes in the other direction, with significantly increased color saturation in Adobe RGB mode. (Average saturation is 120.4%.)
The plot above started out as a mistake, I loaded the Adobe RGB test image, but forgot to tell Imatest that it should interpret the data based on that color space, rather than the usual sRGB. Amazingly, using the DSC-R1's Adobe RGB color space setting produced the most accurate color I've found from any digital camera to date, when interpreted in the sRGB color space! In other words, if you want almost perfectly accurate color from the R1, just shoot with the color space set to Adobe RGB, and then use the images as if they were sRGB. The resulting images will look quite dull compared to the ones shot in sRGB mode, but in fact will be almost dead-on in terms of both hue and saturation. (It's worth noting though, that very few people actually want truly accurate color. Most folks greatly prefer color that's pumped up a bit relative to the original scene.)
Color Analysis
These images show the color behavior of the R1 in its default sRGB color mode. In each color swatch, the outer perimeter shows the color as actually captured by the camera, the inner square shows the color after correcting for the luminance of the photographed chart (as determined by a 2nd-order curve fit to the values of the gray swatches), and the small rectangle inside the inner square shows what the color should actually be, based on perfect rendering to the sRGB color space.
Gray Patch Tone and Noise Analysis
There's a lot in this particular graph, a lot more than I have room to go into here. Bottom line, the R1's noise levels are very low, although the noise spectrum at low ISOs is weighted toward the low frequency end a little. This would produce a coarser noise pattern, but since the absolute levels are so low, the coarser grain structure doesn't really matter.
Here's the same set of noise data at ISO 3,200. The good news here is that the high frequency components actually increase relative to the low frequency ones. Because high-frequency noise tends to be less objectionable than low-frequency, this would normally be very good news. The problem is that the overall level is so high that the images still look awful. At ISOs of 1,600 and below, the R1's images look quite good, but at ISO 3,200, things really fall apart.
This chart compares the Sony R1's noise performance over a range of ISOs against that of other cameras. As you can see, the R1's noise levels are very competitive with those from the digital SLRs on the chart, and completely blow away the performance of competing prosumer all-in-one models. What's not shown here is both the grain pattern of the noise, as well as how the camera trades off noise with subtle subject detail. The R1 does quite well with this latter issue, striking a good balance between reducing noise and preserving subtle detail. Overall, an impressive performance, quite competitive with many of the d-SLRs it competes with.
Dynamic Range Analysis
A key parameter in a digital camera is its Dynamic Range, the range of brightness that can be faithfully recorded. At the upper end of the tonal scale, dynamic range is dictated by the point at which the RGB data "saturates" at values of 255, 255, 255. At the lower end of the tonal scale, dynamic range is determined by the point at which there ceases to be any useful difference between adjacent tonal steps. Note the use of the qualifier "useful" in there: While it's tempting to evaluate dynamic range as the maximum number of tonal steps that can be discerned at all, that measure of dynamic range has very little relevance to real-world photography. What we care about as photographers is how much detail we can pull out of the shadows before image noise becomes too objectionable. This, of course, is a very subjective matter, and will vary with the application and even the subject matter in question. (Noise will be much more visible in subjects with large areas of flat tints and subtle shading than it would in subjects with strong, highly contrasting surface texture.)
What makes most sense then, is to specify useful dynamic range in terms of the point at which image noise reaches some agreed-upon threshold. To this end, Imatest computes a number of different dynamic range measurements, based on a variety of image noise thresholds. The noise thresholds are specified in terms of f-stops of equivalent luminance variation in the final image file, and dynamic range is computed for noise thresholds of 1.0 (low image quality), 0.5 (medium image quality), 0.25 (medium-high image quality) and 0.1 (high image quality). For most photographers and most applications, the noise thresholds of 0.5 and 0.25 f-stops are probably the most relevant to the production of acceptable-quality finished images, but many noise-sensitive shooters will insist on the 0.1 f-stop limit for their most critical work.
The image below shows the test results from Imatest for the Sony DSC-R1.
These results are OK, but on the low end of the scale when compared to many digital SLRs. As usual though, Adobe Camera Raw is able to extract quite a bit more dynamic range from the R1's RAW files (provided that the original image is significantly overexposed; in this case by about two-thirds of a stop), as seen below.
With that as background, here's how the DSC-R1 performed, and how it compares to various digital SLRs that we also have Imatest dynamic range data for. (Results are arranged in order of decreasing dynamic range at the "High" quality level.):
| Dynamic Range (in f-stops) vs Image Quality (At camera's minimum ISO) |
||||
| Model | 1.0 (Low) |
0.5 (Medium) |
0.25 (Med-High) |
0.1 (High) |
| Fujifilm S3 Pro (Via Adobe Camera Raw 2*) |
12.1 | 11.7 | 10.7 | 9.0 |
| Canon EOS-1Ds Mark II (Via Adobe Camera Raw*) |
11.2 | 10.3 | 9.4 | 8.14 |
| Fujifilm S3 Pro | -- | 9.9 | 9.4 | 7.94 |
| Sony DSC-R1 (Via Adobe Camera Raw 3*) |
10.7 | 9.97 | 8.90 | 7.46 |
| Nikon D50 | 10.7 | 9.93 | 8.70 | 7.36 |
| Canon EOS 20D | 10.3 | 9.66 | 8.85 | 7.29 |
| Canon Digital Rebel XT | 10.3 | 9.51 | 8.61 | 7.11 |
| Olympus EVOLT | 10.8 | 9.26 | 8.48 | 7.07 |
| Canon EOS-1Ds Mark II (Camera JPEG) |
10.3 | 9.38 | 8.6 | 7.04 |
| Canon Digital Rebel | 10.1 | 9.11 | 8.47 | 6.97 |
| Pentax *istDs | 10.2 | 10 | 8.87 | 6.9 |
| Nikon D2x | -- | 8.93 | 7.75 | 6.43 |
| Sony DSC-R1 | 9.83 | 8.95 | 7.86 | 6.13 |
| Nikon D70S | 9.84 | 8.69 | 7.46 | 5.85 |
| Nikon D70 | 9.81 | 8.76 | 7.58 | 5.84 |
| * Adobe Camera Raw generally produces the greatest measured dynamic range when the original image is significantly overexposed. (+2/3 stop or more) | ||||
The results shown in the table are interesting. One of the first things that struck me when I initially looked at all the data, was that here again, purely analytical measurements don't necessarily correlate all that well with actual photographic experience. There's no question that the Fuji S3 Pro deserves its place atop the list, as its unique "SR" technology does indeed deliver a dramatic improvement in tonal range in the highlight portion of the tonal scale. I was surprised to see the analytical results place the Olympus EVOLT as highly as they did, given that our sense of that camera's images was that they were in fact noisier than those of many other d-SLRs that we looked at. In the other direction, I was quite surprised to see the Nikon D2x place as low on the listings as it did, given that we found that camera's shadow detail to be little short of amazing.
One thing that's going on here though, is that we tested each camera at its lowest ISO setting, which should produce best-case noise levels. This is in fact what many photographers will be most interested in, but it does perhaps place the Nikons at a disadvantage, as their lowest ISO setting is 200, as compared to the ISO 100 settings available on most other models.
In the case of the DSC-R1, it seems that its dynamic range is on the low side of average, at least based on this particular test. This agrees somewhat with our subjective experience, as it seemed to lose highlight detail fairly rapidly, and shadows showed a fair bit of noise when we tried to pull detail out of them by adjusting tone curves on the computer.
Bottom line though, I think you have to take the figures here with a grain of salt, and look at actual images with your own eyes to see what you make of each camera's tonal range and noise levels. We'll continue performing these dynamic range tests on the digital SLRs that we review, but (just as with the laboratory resolution target results), we suggest that you not rely on them exclusively for making your purchase decisions.
The chart above shows consolidated results from spatial frequency response measurements in both the horizontal and vertical axes. The "MTF 50" numbers tend to correlate best with visual perceptions of sharpness, so those are what I focus on here. The uncorrected resolution figures are 1988 line widths per picture height in the horizontal direction (corresponding to the vertically-oriented edge), and 2067 along the vertical axis (corresponding to the horizontally-oriented edge), for a combined average of 2027 LW/PH. Correcting to a "standardized" sharpening with a one-pixel radius reduced this number quite a bit, to an average of 1572 LW/PH. The uncorrected numbers are indeed very high, while the corrected number is a bit low. Both issues are the result of the slightly heavy-handed sharpening that the R1 applies to its images by default. This produces an unnaturally steep slope in the gradation curve as you scan across an edge in the image, as well as a significant bright "halo" on the light side of the edge, extending out a pixel or two from the edge itself. - See the plots below for a more detailed look at this.
For the real techno-geeks, the two plots below show the actual edge response of the Sony DSC-R1, for horizontal and vertical edges. Here, we can see that the R1's default sharpening is a bit overdone, resulting in a pronounced "bump" on the bright side of the edge. In the main review, I commented on a slight coarseness in the R1's images. With the default sharpening as shown here, we see that the cause of this coarseness is the overshoot in the edge profile, and the distance that it extends away from the edge itself. (On the order of two pixels.)
To avoid this artifact, shoot with the R1's sharpness control set to its "Low" option, and then sharpen your images after the fact on the computer. This will reveal quite a bit more fine detail than you'll see in images shot with the camera's default settings.
Follow Imaging Resource: