Sigma SD9

Test Results

Shoot really compelling product photos for eBay or the web! Visit our free Photo Lessons area and lessons on taking great product shots.

For the full set of my standardized digicam test photos, check out the SD9 Test Photos Page, where you'll find a detailed commentary as well as the photos themselves.

This is much longer than my usual Test Results summary, but several factors dictated the length. The "full color pixel" technology of the Foveon X3 sensor used in the SD9 has captured the public's imagination to an unprecedented degree, and with some justification. It's something genuinely new in the digicam market, and the concept and its apparent advantages are easy to explain and understand. This has led to a level of excitement in the digicam community beyond anything I've seen previously, with claims and counterclaims abounding, fueled by a paucity of hard information. Many enthusiasts have decided that the SD9 should be their next camera, based entirely on the strength of the concept, rather than objective evidence its performance. Discussions on the 'net regularly devolve to the level of religious dogma, and few people seem to be taking a dispassionate look at what the technology in general and the camera in particular is really capable of.

In the face of this, I wanted to be careful to describe the benefits and shortcomings accurately and completely. I don't plan to make this level of discussion a habit in my camera reviews (I don't remotely have the time), but the level of communal agitation produced by Foveon and Sigma seem to call for it in this instance.

For the full set of my standard test shots, as well as my detailed comments thereon, see the _SD9's Pictures Page._ For a set of more photographically interesting subjects, shot with four competing SLRs, see our _SLR Shootout Page._ For a condensed summary of my findings and some more general comments on the camera and sensor, read on below.

My initial reaction to the SD9 was generally pretty favorable. It seemed like a capable camera, offering high resolution and good color at an affordable price. The more time I spent with it though, the more I found some of its behavior problematic, leading me to revise my overall opinion of it somewhat. Here's a summary of some of my findings, good, bad, and otherwise.

User Interface
I'll start out with something positive. I liked the SD9's user interface quite a bit. The one-function-per-button design is both very easy to master, and fast to use in practice. The rear-panel LCD menu system is very clear and easy to navigate. I particularly liked the Sigma/Foveon implementation of the histogram display, easily superior to any other I've seen. All in all, it's a convenient camera to use, quite easy to get used to after only a little shooting with, and quick to navigate while shooting.

Resolution
Foveon claims that the X3 sensor technology delivers twice the resolution per pixel of conventional striped sensor arrays. By that measure, the 3.43 (effective) megapixel chip in the SD9 should deliver resolution early equivalent to that captured by 7-megapixel conventional sensors. In my tests though, the SD9 actually resolved slightly less detail than its 6-megapixel competition. Images from the SD9 are very sharp, thanks both to the sensor technology itself, but also to the lack of an anti-aliasing filter. This may lead some to regard the SD9's photos as being "higher resolution" than those from its 6-megapixel brethren, but the bottom line is that it captures slightly less picture information. Lacking an antialiasing filter, it is also more prone to luminance aliasing. While not subject to the sort of color aliasing seen in striped sensors, the X3 sensor is still a sampled-data device, and so is prone to jaggies along sharp contrast boundaries, and moire patterns in fine-grained, repeating detail.

Aliasing
Having just mentioned aliasing, I want to spend just a moment in an attempt to clear the air on aliasing, as there's been a tremendous amount of confusion on the 'web lately about this. In particular, because artifacts arising in aliasing can look like subject detail on resolution target images, some people have been misled into thinking that the sensor was actually imaging some (but not all) of the target lines. In actuality, what was appearing in the image at that point was a pure illusion, in no way representing the original image data. The enlargement at right of an SD9's shot of a resolution target element shows what I'm referring to. At the very bottom of the hyperbolic resolution wedge I've cropped out,four or five vertical lines do indeed appear, so it's easy to see why some would interpret these as being representative of the target lines. Consider though, that none of these lines correspond in either size or position to the lines of the target itself. As further evidence, look higher in the image, and observe the patches of gray that seem to "float" around on the image. There's clearly nothing in the original target to correspond to these, but they nonetheless appear in the final image. Just to state it clearly, aliasing is always a bad thing, because it introduces spurious information into an image that doesn't reflect the subject detail (and in most cases, actually obscures it).

The misconception seems so widespread, that I feel I have to say this again: It is absolutely incorrect to say that the SD9 is "resolving detail above it's resolution limit" or words to that effect. What people are seeing is not scene detail, but rather just artifacts that they're mistaking for such. (In a past life, I specialized in image processing computer architectures and algorithms, so it makes me grate my teeth to see people looking at image artifacts and thinking that they're a good thing to have.)

Now, if aliasing is (apparently) so bad, it's natural to ask why Sigma and Foveon would design a camera that is so prone to it. The answer is that there's a tradeoff involved, and the other side of that tradeoff is a very desirable increase in edge acuity and sharpness. By leaving out the antialiasing filter, Sigma and Foveon produced a camera with very abrupt transitions across contrast boundaries in the image, something that our eyes interpret as "sharp". The downsides are aliasing, as we saw above, and also "jaggies" along gently curving or slightly angled edges. But some people may not be bothered by jaggies, and end up liking the "look" of the SD9's images in spite of them. Likewise, aliasing will only be apparent when dealing with finely-spaced, high-contrast detail, as with some man-made textures, particularly certain fabrics. If you don't shoot a lot of that sort of subject, you may not encounter the aliasing problem at all. (And interestingly enough, both problems will be reduced if you're shooting with a lower-quality lens with less resolving power than a pro-grade one. The lens itself can act as an antialiasing filter. - So if you cheap out on lenses for your SD9, you may actually end up with smoother-looking photos.) The bottom line is that this is going to come down to a matter of personal preference. Some people will like the very crisp-looking images that the SD9 produces, and either not be bothered by or be willing to accept the jaggies and aliasing that come along with it. Others will find the artifacts objectionable enough that they're unwilling to accept them in exchange for the sharp images.

Autofocus
This appears to be one of the SD9's problem areas. Autofocus speed will obviously depend on the particular lens being used, but overall the SD9 is no speed demon in this area. Worse, it can sometimes fail to focus altogether, even with well-lit subjects. Our pro shooter Gibbs and I both saw this, although not frequently enough to be able to fully determine what triggered it. The problem seems to occur when the focus mechanism has to traverse a large part of its range when shifting from one subject to another. - I know I saw it several times when using the 50mm macro lens, which has a very broad focus range, but I think Gibbs saw it both with that lens and the 20-40mm zoom. What happens is that the lens will rack back and forth, "hunting" for the correct focus setting, but never actually find it. The camera eventually seems to "give up," stopping at some intermediate position far from the correct focus point. At least once for both of us, continuing to hold down the shutter button led to the camera waking up again after a few seconds, and eventually finding the correct focus position. This was disconcerting behavior to say the least, and is something I've never before seen in a digicam, except under low-light conditions, which was not where both Gibbs and I encountered it.

Exposure
I found the SD9's exposure system to rather unpredictable, often underexposing the subject significantly. With low-contrast subjects, it usually produced correct exposures, but even then sometimes got it quite wrong. (The "Musicians" poster was one example of this. - This isn't by any means what I'd consider a high contrast or difficult subject to meter, but the SD9 consistently underexposed it by a full f-stop.) When dealing with high-contrast scenes, the SD9 almost always underexposed the subject, apparently trying to hold onto highlight detail, even when the highlights covered a very small portion of the total frame area. This was a particular problem when shooting night scenes with small bright lights included in the frame. - I found I had to dial in as much as +3EV (3 f-stops) of exposure compensation to get a reasonably bright image in such situations, while competing SLRs required far less.

Tonality
The SD9 tends to produce rather contrasty images straight from the camera. This no doubt is part of the reason for its bright, saturated color: Pumping contrast always helps color saturation. The high contrast can cause problems with highlight detail and shadow noise though, when you try to compensate for it post-capture in the Photo Pro software. Worse, the high native contrast means that even the raw image data often lacks detail in the highlights. (See my comments on post-capture exposure correction below.)

Color
Color was a bit of a mixed bag, but generally good. Under daylight-balanced lighting, colors were rich and vibrant. (No doubt a bit too saturated for some photographers, but that's easily handled in the Photo Pro software.) Hues are generally accurate, and the only weakness I observed was a tendency to undersaturate greens a bit. Color was a bit less well-behaved under imperfect lighting conditions, with high-contrast subjects sometimes resulting in lower color saturation in the final image files.

Color loss in highlights (and other findings by Phil Askey)
I can't take credit for this one - I noticed the effect in shots I took with neon signs in them, but didn't tumble to what was actually happening until I read my friendly competitor Phil Askey's excellent review of the SD9. Phil correctly identified the issue as being that the camera drops the chroma (color) to zero as it runs into the highlight range. Phil also identified a number of other image-quality problems on the page of his review linked above, including unnaturally blue skies, strange halos around light sources in night shots, some odd chromatic aberration that seems to be linked to UV light, image softness and loss of saturation at high ISO, and an odd nighttime exposure bug. Definitely worth reading if you're considering an SD9 for purchase.

White Balance
White balance was a bit of an Achilles heel for the SD9. It worked quite well in lighting that was more or less daylight-balanced, but had quite a bit of difficulty with light having strong color casts. The most prominent example of this in my own shooting was our "indoor portrait" test, shot under household incandescent lighting. This shot has proven to be a pretty harsh test of digicams' white balance systems, as it's a *very* warm-toned light source. - The conventional wisdom in the photo industry is that household incandescent lighting has a color temperature of 2800K (vs the 3200K of professional tungsten and the roughly 5500K of standard daylight). By my actual measurement though, household incandescent is generally closer to 2300K, a very yellow hue. The SD9's auto white balance couldn't correct for this strong a color cast, leaving a fair bit of yellow in the image. The SD9's custom white balance resulted in greenish casts under this light source, and in fact in general produced noticeable color casts in most of my tests. Most cameras have difficulty with the incandescent lighting in their automatic white balance modes, but competing SLRs do better than the SD9 when using manual or custom white balance settings.

Image Noise
There seems to be a fair bit of confusion and controversy over the SD9's noise performance. As is always the case, some of the noise is doubtless attributable to the X3 sensor itself, while some of which likely has its origins in the SD9's signal processing circuitry. - This means that at this point, we can't fully determine how much noise is a consequence of the X3 technology, and how much might be caused by electrical noise in the SD9's signal-conditioning and -processing chain.

The noise controversy has doubtless been further fueled by the fact that the noise varies quite a bit depending on the subject, being a fairly strong function of subject color and saturation. I'm making some in-depth noise measurements of the SD9 and several other SLRs, for later presentation on this site, but my initial results have been illuminating. In neutrals (shades of gray), noise at low ISO is actually fairly close to that of competing SLR models, perhaps an average of 20% or so higher. (That's a very ballpark figure though, so don't hold me too closely to it when my more detailed analysis comes out. The bottom line is that it's clearly higher, albeit not by a large amount.) Where noise really skyrockets though is in the red/magenta portion of the spectrum. In that range of colors, the SD9's noise levels are as much as 2-3x higher than competing models. At higher ISOs, the SD9's noise performance appears to lag that of other SLRs, but I haven't done the data reduction yet to be quantify that.

Software
A significant feature of the SD9 is the Foveon-developed Photo Pro software. This is clearly one of the best post-capture image adjustment programs I've yet used, with a very "photographic" sensibility. - By that I mean that I think photographers will find it very natural to use and easy to learn.

I do have one piece of feedback for Foveon on Photo Pro though, which is that I'd like to see a more continuous range of control in its color balance adjustment. I often found that moving from a correction of, say, 5C+4M to 6C+4M produced a larger increment of color shift than I wanted. - A "5.5C" setting or its equivalent would have been nice. In saying this though, I don't want to detract too much from Photo Pro, as it's an exceptionally effective and intuitive tool for adjusting color and tone in photos.

I was less impressed with Photo Pro's "auto" correction capability though. It worked pretty well for images that were close to the correct exposure and with modest color casts. If the exposure was off by more than half an f-stop though, or if the subject was too contrasty, Photo Pro's automatic adjustment rarely resulted in what I'd consider a usable photo. Given the SD9's mediocre white balance performance, this means that many photos are likely to need manual adjustment. Not a problem if you enjoy tweaking photos on the computer, but for many users the need to tweak significant numbers of their photos could be an impediment.

RAW-only File Format
This is both a strength and weakness of the SD9's design. For all the conceptual simplicity of the X3 technology, it apparently requires some pretty hefty number crunching to unscramble the data in the color channels. This means that it's not feasible to process the images to a final JPEG format on the camera, hence the SD9's RAW-only file format. While Photo Pro does let you batch-extract your images to TIFF or JPEG, as noted above I felt that I needed to manually tweak most of the images I shot, negating the benefits of batch processing.

Of course, another factor with RAW-format files is that they're a lot bigger than even high-quality JPEGs. This translates into both greater storage requirements (a need for larger memory cards) and slower buffer-clearing times when shooting series of photos. - The SD9 took a really long time to clear its buffer to the CF card, after shooting a set of 6 images. As to card size, you should consider a 256 MB memory card as the bare minimum to work with, especially if you're at all inclined to bracket your exposures.

Post-Capture Exposure Correction
One benefit that's held out for RAW-format image files is the ability to correct for over- or underexposure after the fact. The idea is that the file contains 12-bit (or at least some bit depth greater than the eight bits/channel in the final file), so you should be able to select the "best" 8 bits for final output. Indeed, in the professional realm, Kodak has made quite effective use of this capability as a marketing tool for their high-end SLRs. Unfortunately, just having a RAW file format doesn't automatically mean that it has sufficient data in it to permit this sort of post-capture exposure manipulation.

In the shots I snapped of my standard test subjects, I found little or no "hidden" data present in the SD9's RAW files, at least in the case of blown highlights. This seemed odd, as other reviewers have claimed that they've seen a full f-stop of detail that could be recovered from blown highlights, where I saw essentially none.

Given this discrepancy, I to test the exposure headroom of the Sigma/Foveon RAW files directly.

The approach I took was quite simple. I made a test target in Photoshop, consisting of a couple of density ramps that faded off into pure white. To provide a more subtle gray-on-gray measurement (rather than gray against the stark white of the paper), I put a second ramp inside each primary one, with the density values of the secondary ramps adjusted so as to produce a fairly consitent step between them. The resulting target (see the illustration photos below) isn't calibrated to any absolute standard, but it turns out that it really doesn't need to be for the sort of rough measurement we're doing here. - All we need to do is to compare the point at which the camera loses the grayscale with various combinations of exposure and post-capture processing.

A simple test target for observing exposure headroom in RAW files.
Nominal Exposure	Moderate Overexposure	Strong overexposure

The table above shows what the target itself looks like, and what happens when it is overexposed to varying degrees. Note how you can clearly see where the visible highlight detail stops in each image, and how the point at which it stops moves lower the more the target is overexposed.

As it turned out, this approach worked very well for testing the SD9's RAW-file exposure headroom. What's more, it completely confirmed the informal impression I had developed from looking at my standard test shots. - It turns out there's only a quarter-stop or less of headroom in the SD9's raw images, certainly not an amount that's useful photographically. (Keep in mind that the total overexposure below is only a half an f-stop.)

The images below show crops from shots of this target taken at two different exposures and with different post-capture treatments. Note that the way I conducted this test was actually quite generous to the SD9, in that I wasn't judging it by whether I could reproduce a nice-looking approximation of the default tone curve in the overexposed images, but instead was simply looking to see at what point it could pull out any detail from the images. To see this, I jacked the tone curve around quite a bit with the Levels control in Photoshop, so I could see just where the first hints of data began to appear. In the table below, I've included both the original images and the ones that I've adjusted in Photoshop. I've also included links to TIFF-format crops from the TIFFs I extracted from the RAW files, and notes on the settings I used in Photoshop's Levels control to produce the tweaked images, so everyone can see exactly what I did.

Results of Exposure Headroom Test
Default exposure ISO 100 40mm lens f/11 1/10 second ~4800K lighting	Plus 0.5 EV ISO 100 40mm lens f/11 1/8 second ~4800K lighting	Plus 0.5 EV, adjusted in PhotoPro ISO 100 40mm lens f/11 1/8 second ~4800K lighting Adjusted -1.0 EV in PhotoPro
As above, but adjusted with Photoshop's "Levels" control to show point at which highight detail stops. Shadow = 201 Highlight = 252	As above, but adjusted with Photoshop's "Levels" control to show point at which highight detail stops. Shadow = 234 Highlight = 253	As above, but adjusted with Photoshop's "Levels" control to show point at which highight detail stops. Shadow = 194 Highlight = 229
Original TIFF File	Original TIFF File	Original TIFF File

As you can see, the PhotoPro software only managed to pull back the tiniest amount of the highlight detail that was lost when the exposure was increased by only 0.5 EV (1/2 of an f-stop). In the middle example, extracted with PhotoPro's default settings, the detail stops just below line 5. (What looks like detail above that point is actually just a variation in noise patterns. You can tell that there's something there, but the differences you see there amount to only one brightness unit in the file itself. They're visible here because of the extent to which I've cranked on the Levels adjustment to make the cutoff point visible.) In the example on the right, where I've pulled the exposure down by a full f-stop in PhotoPro, the detail stops just under line 4, althought the same sort of noise patterning shows the presence of the target pattern one level above that. (For those who may be wondering, I pulled the exposure down by a full stop in PhotoPro to be sure that I'd be able to see anything at all that was in the RAW file to be found. While this made the overall image darker, including the pure white areas, this assured me that any information that might be present would be moved down into the active tonal range.)

Given that a half an f-stop of overexposure produced a highlight loss of between four and five divisions on my makeshift chart, the roughly one division's worth of highlight that I managed to bring back from the RAW file amounts to less than a quarter of an f-stop of recoverable data.

So why have people been claiming that there's a full stop's worth of headroom in the SD9's RAW files? I think it's simply a matter of not having looked carefully at the data that was actually present in the images extracted with PhotoPro's default settings. What I think has been happening is that manipulations in PhotoPro are simply bringing detail that was present in the default images down into a tonal range where it becomes easily visible on-screen. There data was right there all along, and could have been seen simply by tweaking the tone curve in Photoshop without going back to the RAW file at all. Thus, while it looked like more detail was appearing, it was simply a case of detail that was already there being made visible. My guess is that nobody until now has thought or taken the time to do the sort of comparison shown above.

Given the controversy that I expect my findings to create, I've provided crops of the TIFF images extracted from the RAW camera files for anyone who'd like to play with them in Photoshop to see for themselves what data is or is not present in them. Simply click on the links below each image above to download these TIFF files to your local hard drive. In most cases, your browser won't know what to do with files with .tif extensions, so will simply ask you if you want to download them or not. (And before I get the inevitable emails about them, those spots you see on the images are from dust particles on the SD9's sensor surface, not blotches on the target itself.)

Phew... OK, on to one remaining issue, and then my conclusion:

Dust and Quality Control
This was a real shocker, but at least ought to be able to be fixed with a bit better control of Sigma's manufacturing environment. The sample SD9 I received was laden with dust inside the body (behind the dust screen), clearly visible on the mirror, and painfully evident in photos shot with the lens stopped down. (I'm pretty certain I was the first reviewer to comment on this.) As noted, this should be easy for Sigma to correct, but the fact that the dust was there in the first place is a little troubling.


Conclusion

As I said above, my initial reaction to the SD9 was pretty positive. Now that I've had more time to work with it though, some of my enthusiasm for it has waned. In particular, it has obvious issues with its autofocus, and I found its exposure metering to be problematic as well. It's white balance also struck me as less effective than that of its competition, and the camera's maximum 400 ISO and limited maximum exposure times further restrict its utility.

The SD9's X3 sensor does indeed deliver more resolution per pixel than conventional striped sensor arrays, but it still doesn't resolve quite as much detail as the six-megapixel competition. Its images are very sharp, but that's at least partially because it has no antialiasing filter. Thus, while it is pretty completely immune to color aliasing, it is much more subject to luminance aliasing than its competition. It's possible that including an aliasing filter in the design would indeed show the X3 sensor delivering 2x the resolution per pixel of conventional arrays, but as it is the strong luminance aliasing obscures the finest detail.

The combination of full-color pixels and lack of an antialiasing filter clearly produces images with a different "look" to them though, so there will doubtless be shooters who will prefer its images to those of other camera models. - Raw performance measurements rarely translate well into personal preferences.

The SD9's exposure flexibility is quite good, with all the modes and controls you'd expect to find on an SLR, although its 0.5 EV exposure adjustment steps are a little coarse for a digicam. Its combination of a RAW file format and the excellent Photo Pro software provides an excellent ability to adjust images post-capture, but at this point I have to say that the jury is still out as to whether the RAW files actually contain significant highlight detail beyond what appears in images converted to JPEG using the default settings.

The Sigma lenses that couple with the SD9 body do indeed appear to have very good optical characteristics, as well as very competitive prices in the marketplace. This could be viewed as a reason to buy the SD9, for those interested in accumulating a large collection of lenses on a budget. On the other hand, most of the same Sigma lenses are available for roughly the same prices with Nikon or Canon mounts, so anyone would be free to use them with one of the competing SLR designs if they wished.

I commented in the conclusion to my original "first look" at this camera that I expected something of a holy war to erupt on the 'net over the SD9's color rendition, resolution, and image noise, and that indeed has come to pass. It does appear though, that the images I and others have posted from the camera have injected a healthy level of reality into the discussions, and I hope that the test shots I've now posted, as well as the raw-format highlight headroom results (and eventually, some truly detailed noise measurements) will continue that trend.

As the first digital camera from a company not previously involved in the digital world at all, the SD9 is a remarkable achievement. It clearly has a number of limitations though, some of them serious. Taken as a whole, I think users give up an awful lot of capability for the roughly $400 in price that separates it from the competing models from Nikon and Canon (and the roughly $800 diference relative to Fuji's S2 Pro). I do think though, that the SD9 will nonetheless find happy homes with many shooters drawn by that price difference, and by the different "look" of the images produced by its full color pixels and non-antialiased optical design. Ultimately, the decision to purchase an SD9 will be a very personal one - I hope that the information I've presented here will help people make it in an informed fashion.