The Canon 50mm f/1.4 USM: a multi-sample evaluation

Updated February 19, 2010: We've added some additional reading relating to other variation sample testing done on an external site, diglloyd.com. To skip to this new section, click here.

Updated February 5, 2010: Dave's added some commentary about the practical impact of this test. To skip to this new section, click here.

Introduction

As with most test organizations, the combination of time and resource constraints and relatively limited access to the review samples means that most of the time we can only test a single sample of each product, be it a lens or camera. Modern manufacturing is generally excellent at cranking out identical copies of a product, but the extraordinarily close tolerances involved in optical manufacturing mean that there's likely some noticeable variation from lens to lens. It's long been the case that photographers with deep pockets and lots of patience could often improve their lens kit by buying and testing several copies of each lens, selling on the lesser samples. The Internet is full of forum posts and web pages with people discussing lens sample variation, and more than a few examples have been posted, showing very noticeably different results from identical shots taken with different copies of the same lens. To the best of our knowledge, though, no one has thus far subjected multiple samples of lenses to rigorous, quantitative testing.

Sample variation is a significant issue for users, reviewers, and manufacturers. As a user, the issue is whether a positive review and glowing comments by other users mean that you'd have a similar good experience with a lens should you purchase a copy. On the flip side, do a few negative reports about a lens' performance mean you should avoid it, or are they merely flukes, with the majority of samples actually performing well?

 

5 copies of the Canon EF 50mm ƒ/1.4 USM: serial numbers #60302489, #54323497, #53207976, #53120225, and #50102461.

In this article, we will provide the results of a comparison test between five samples of the same lens, in this case, the Canon EF 50mm ƒ/1.4 USM. It's been often commented that the results of testing just one sample of a lens can lead to misleading results, as often there are better and worse samples of a given lens. To attempt to assess the impact of this claim, we have partnered with LensRentals.com to test several copies of the same lens.

Canon EF 50mm ƒ/1.4 USM

The Canon EF 50mm ƒ/1.4 USM hit the market in June of 1993, using a design featuring seven lens elements in six groups. The aperture is made up of eight diaphragm blades, and the lens takes 58mm filters. With USM focusing, autofocus is fast and near-silent, and autofocus results can be overridden at any time by simply turning the focus ring.

The lens was designed originally for film cameras, making it fully compatible with Canon's full-frame camera bodies. On a cropped-frame camera body, the effective focal length is 80mm. The aperture ranges from the maximum (widest) setting of ƒ/1.4 to the minimum (smallest) setting of ƒ/22. The lens is currently available for approximately $400.

Sharpness

In the following interactive graph, you'll see the five samples of the 50mm ƒ/1.4 USM, labelled from "A" to "E". We've actually decided to expand the "ceiling" of our graph from 12 Blur Units to 20 Blur Units to accommodate the unusually high blurring on the right side of the frame.

There are definitely differences between the samples, in terms of just how soft the corners are wide open, and to some extent how lopsided or symmetric the blur characteristics are. The worst corner softness is on sample #53207976 ("C"), the best (although still fairly extreme) is on sample #53120225 ("D"). All have a very similar general appearance, though, with the right side of the frame much softer at ƒ/1.4 than the left, but the center for the most part fairly sharp. On sample #60302489 ("A") the sharpness profile was noticeably different, in that it had a sizeable lump (softer spot) in the center left of the frame at ƒ/1.4. We've seen very "lumpy" behavior like this in the past from some large-aperture lenses (notably the much earlier version of this lens that we first tested several years ago), but this one was interesting because it stood out from the others in the batch in this regard.

As you'd expect, stopping down improves the optical performance, although at ƒ/2, the corners remain quite soft on all samples. Sample number #53120225 ("D") again does better than the others, bit the odd lump in sample #60302489's ("A") blur plot disappears entirely. Interestingly, while the corners improved significantly from where they were at with ƒ/1.4, they remain very soft - but the center actually becomes quite sharp, and the size of the sweet spot increases noticeably.

There's more improvement at ƒ/2.8, but corners on a couple of the lenses are now noticeably softer than those on the others: sample numbers #50102461 ("E") and #53207976 ("C") are notably softer in the upper right corner of the frame - Perhaps not surprising, since these were the two most asymmetric wide open, but it's interesting the extent to which the others have settled down, while these two still retain their corner problems.

By ƒ/4, the lenses all look pretty similar, with pretty good sharpness across the whole frame. Sample number #53120225 ("D") still leads the pack, but do note that the blur differences represented by the differing shades of purple are quite slight. From ƒ/5.6 onward, all five lenses are close to identical in their characteristics: not perfectly uniform sharpness across the frame, but pretty close, and also pretty sharp.

We were a little surprised to see that all five samples were noticeably soft on the same side: we'd somewhat expected that if there were asymmetry, we'd see variations in that asymmetry, with some samples being sharper on the right, some on the left, etc. Our setup and test body were the same as we've been using for our tests for some time, and we haven't seen any consistent softness on the right side of the frame before, but just to make sure nothing was amiss in the setup, we attached a 50mm ƒ/1.8 lens onto the camera and checked its performance at ƒ/1.8 and ƒ/2. Fortunately, the results matched our expectations: the 50mm ƒ/1.8 lens itself had asymmetric blur at wide apertures, but rather than being soft on the right side, it was softest at the bottom, and relatively balanced left to right.

Chromatic Aberration

Looking at other parameters, we find that CA and geometric distortion were nearly identical across all 5 lenses. There are some slight variations in vignetting, but all of the lenses were very close to each other in this parameter.

Corner Shading

 

 

Distortion

 

 

In comparing the results of corner shading ("vignetting") and distortion between the five lenses, we did not note a significant difference between the results from each.

Sample Images

We've taken sample images for one of the lenses, sample #53120225, so you can see exactly how the results look in real life.

 

Our previous test of the Canon EF 50mm ƒ/1.4 USM.

Summary

We tested a sample of the Canon 50mm ƒ/1.4 USM several years ago, with rather different results: the sample we tested back then also showed soft corners at ƒ/2, but flattened out amazingly well a ƒ/2.8, yet was incredibly soft and uneven at ƒ/1.4. Wide open, none of the frame was sharp, and the sweet spot (if it could be called that) was a C-shaped area from the middle right to middle bottom of the frame. We'd have suspected impact damage or other problem were it not for the blur being so symmetric a ƒ/2, and so well-controlled overall at ƒ/4. Looking back at the earlier raw data, while it was collected prior to our use of the optical rail for focus bracketing, our results from the manual bracketing we did at the time appear self-consistent and valid. Our only conclusion is that Canon has made design or manufacturing changes in the 50mm ƒ/1.4 USM since the lens we tested then was made. Unfortunately, in those early days, we didn't routinely record lens serial numbers as we do now, so there's no way of knowing just when that earlier sample was manufactured. It seems probable that it was made some time earlier, since we obtained from Canon Professional Services here in the USA, where it had likely been in the loan pool for some time.

Given that five current samples spanning a modest range of recent serial numbers showed such a strong resemblance to each other, we'll be updating our review of the 50/1.4 with results from the best of the current samples, #53120225 ("D").

Overall, we were a little surprised not to find more differences between current samples of the Canon 50mm ƒ/1.4 USM, although there were clear differences among the five we tested. We were quite surprised by how different the current samples were from the unit we originally tested, which we're guessing was at least 5 years or so old.

Practical Impact

Since posting these results, we've had a lot of emails from readers wondering at the high blur scores when shooting wide open, and saying that if blur in the sides and corners of the Canon 50mm ƒ1.4 lens were really as bad as the DxO scores indicate, there would have been a huge hue and cry over it by now, from unhappy owners.

In fact, we think that many samples of this lens out there show the kinds of things we saw in the five samples we tested, but they may not be apparent to many owners, given typical usage. Here's a brief bullet-point summary detailing why we think this is likely the case:

• The distortion in the corners/edges is a very odd "smearing" that we don't believe we've seen before in other lenses. In the corners, dark areas are smeared into bright ones, as a sort of a ghost, as can be seen in one of the crops in the window above. It's not a general softening per se, but a very directional smearing, and also a good bit fainter than the main body of the dark detail. In a blur measurement, the extent of this smearing produces high blur scores, but detail at some angles is still well-preserved.
• Because the blur is directional, there'll still be a good bit of detail present in most natural scenes, so people may be less likely to notice this particular sort of blurring.
• As a side note, the direction of the smearing leaves near-vertical edges relatively distinct. This means that testing approaches employed by some other sites that use near-vertical slanted edges would miss it almost entirely, because the edges they're testing along are in a direction that will be relatively unaffected. (So it won't show up on lens-sharpness plots using this methodology.)
• Given the broad extent of the smearing, and that it is a good couple of stops up the tone curve from the dark detail producing it, most users will experience the smearing more as a loss of contrast in the corners of the frame than as a loss of resolution per se. (You aren't likely to see the smearing directly unless you have strongly contrasting elements against a relatively clear field, as in test charts. Natural subjects will look a lot like the contrast is just lower.)
• This lens also vignettes fairly heavily in the corners, so the lower brightness will tend to make the smearing less evident, unless you're correcting for the vignetting in Photoshop or other software.
• Apart from architectural photographers (who are less likely to be shooting at ƒ/1.4 anyway, as they more often want plenty of DOF), it appears to be the case that people shooting at wide aperture are often less interested in detail in the corners of the frame than elsewhere. - We went looking on the 'web for images shot at ƒ/1.4 with this lens, and almost without exception, the shots we came up didn't have any detail in the corners to speak of: Typically, the content in the corners was background the photographer was wanting to drop out by shooting at such a large aperture, or the plane of focus was otherwise such that it was hard to find anything of interest in the corners or even near the edges of the frame.

Checking back to some of the files from our original test of this lens 3-4 years ago, we see some of the same behavior as found here, but not as pronounced as in the more recent 5 samples. - So the same artifacts were present, but not being as extensive, they didn't produce blur scores as high as in the current samples.

In the case of the Canon EF 50mm ƒ/1.4 USM, the way in which its images become soft at the corners produces high blur scores, but given that most shots at ƒ/1.4 aren't set up with strong, high-contrast detail in the corners, it's easy to see why there haven't been more complaints about the phenomena. Times when people really care about corner detail, they're more likely than not also shooting at smaller apertures, where the lens actually becomes quite sharp across the frame.

Other experiences with sample variation

Our experience with lens variation is far from unique; you can find references to it all over the Internet. Of particular note, though, are Lloyd Chambers' observations on the subject: Lloyd does some of the most thorough, rigorous experience-based lens tests anywhere on the 'net, so his reports carry particular weight. Most of his content is available only via subscription (a very worthwhile subscription indeed, we might add), but he has a free article posted on the topic that make for interesting reading:

• Brand-new blur

To show that it's not just Canon or Nikon lenses that are subject to sample variation, Lloyd's also published an article on variation in Zeiss ZF/ZE lenses, although these are only available via subscription:

• Canon 100mm f/2.8L: Brand-new blur
• Zeiss ZF/ZE Sample Variation

The Nitty-Gritty details

Finally, if you would like to pore over the details for each lens, we have posted special pages just for you: