Sigma’s key to quality? Our exclusive look at the A1 system reveals all!

 

 

One thing that's unique about Sigma's recent line of "Global Vision" lenses is that every single lens is subjected to exacting, computer-driven optical tests to ensure it meets its specs. Sigma developed their own test system, using the same high-resolution Foveon sensor chip found in their SD1 digital SLR. It's such a key part of their production process and represents such a unique end-user benefit in terms of reduced sample variation, though, that Sigma has kept details of the system very close to their vest.

Until now.

Back in August of 2013, I had the unique opportunity to tour Sigma's factory in Aizuwakamatsu, Japan, and report in considerable detail on what I saw there. That level of access to a Japanese manufacturer's production line was unprecedented, in my experience. Perhaps partly as a result of our care with the sensitive imagery and information from that tour, I managed to persuade Sigma's CEO Kazuto Yamaki to give me a detailed look at their A1 testing system when I visited him again in Aizu after the CP+ 2015 show in Yokohama in February. I can't share all that I saw, but Yamaki san was quite generous in allowing me to show photos of the system and discuss many details of its operation.

It's an interesting system, and remarkable that Sigma uses it to test all of their Global Vision lenses coming off the production lines. I believe some camera and lens manufacturers test some lens models 100%, but to the best of my knowledge, those tests are done with optical projection systems, so there's still a slight degree of variable human element in determining the pass/fail thresholds. With the A1 system, acceptance thresholds are specified in terms of absolute, measured MTF numbers, at various points across the lens's field of view. As far as I know, Sigma is the only company in the industry to do this.

So here are some pictures, images, and observations about Sigma's A1 test system. One of the most impressive things? Sigma has no fewer than 60 of these systems deployed on their production lines and continues to churn out more of them, week by week. (It takes about two weeks to build each A1 system, from start to finish. I don't know whether they assemble more than one at a time, though.)

 

Here's a shot of the A1 system and its proud creator Masatoshi Ishii. Mr. Ishii worked as a lens designer at Sigma's headquarters prior to developing the A1 system. Once the A1 system went into production use, he moved to Aizuwakamatsu, and now serves as a liaison between the factory and the lens designers.

Here's a look at the overall system, with its proud creator, Masatoshi Ishii, Section Chief, Production Engineering Division (he's what I think we would call in the US a "production engineer") standing beside it. The A1 system uses a standard slanted-edge target to measure MTF (modulation transfer function, one measure of optical resolution) at multiple points within the frame. Separate sharpness thresholds are set for the center and corners of the frame, as well as for the ratios between sagittal and axial values; falling short in any of the five areas, in either overall MTF, or too much spread between the sagittal and axial measurements causes a lens to be sent back for rework, to see if it can be adjusted to spec. If it can't be brought into spec, it's scrapped. (Oh the agony! Can you imagine throwing out a thousand-dollar-plus lens because one of its corners is just a little shy of the standard? That's the strength and the pain of 100% digital testing; the computer will never say "well, this one's pretty close, maybe we can let this one slide..." A lens either passes or it doesn't.)

Sigma had a digital lens-testing system prior to the A1, but it was based on lower-resolution (and expensive) Kodak sensors. While neither Yamaki san or Ishii san recalled the specific details of the sensor used, they were actually smaller than APS-C size, so five of them were required to cover the full-frame image circle. Also, because they were Bayer-striped designs, they had lower spatial resolution than the Foveon chip used in the current system.

Sigma logs all the test results from every Global Vision lens in a database, and that data is available to customer support staff worldwide. So if a customer calls in, saying "hey, my lens is soft on the left side," they can check the database to see whether it might be a case of a lens that was at the edge of being in spec and may have drifted over time, or if there had been a drastic change (indicating that something might have happened to the lens, either in shipping or use). 

I was curious whether they might also archive some of the actual target images from the testing process, perhaps in the interest of performing any longer-term analysis of manufacturing yield, but they said no, the distilled-down numbers are all they need, for either purpose.

 

The business end of the Sigma A1 lens testing system, with a Canon-mount Sigma 24mm f/1.4 Art lens attached. As you might expect of a device that needs to hold such close tolerances, the unit is very rigidly constructed from thick metal plates. The cable you see trailing below the lens lets the control box at upper right "talk" to the lens, via the contacts on its mount.

One thing I found particularly interesting was that the A1 system routinely brackets focus slightly around the optimal setting. It turns out that this gives more insight into possible decentering problems. By checking best focus, a slight back-focus and a slight front-focus, the system gets a better picture (no pun intended) of possible decentering issues. With just a single result for the point of best focus, they can tell whether a lens meets the specs or not, but looking at the MTF data for slightly front- and back-focused settings, they can tell more about what's going on. At a basic level, if the MTF values on one side or the other of best focus are significantly different, that's an indication that there's some decentering at work. (It wasn't stated, but my impression was that the particular way things might go out of focus might be a clue as to which elements in the optical formula were decentered.)

When it comes to zoom lenses, the production line test checks the lens at its shortest and longest focal lengths. The rationale is that if it performs properly at the extremes, it's unlikely that there would be a problem in between. 

Sensor movement; covering full-frame
The A1 system has been in use for about four years now, basically since the advent of Sigma's Global Vision lens line. Ishii san said it took about a year to develop, once they had the initial idea. One of the biggest challenges with the system was that the SD1 sensor was APS-C sized, but they needed to be able to test full-frame lenses as well. That meant they needed to be able to move the sub-frame sensor, to be able to cover the corners of the full-frame image circle, as well as the center. The catch, of course, is that they had to move the sensor around the larger frame while maintaining micro-level alignment with the lens axis. Ishii san didn't describe the solution in detail, but the general idea is that they use very tight-tolerance mechanical components and have adjustments built into the system that let them achieve perfect alignment. Sigma already had incredibly precise digital measurement systems, capable of measuring dimensions with sub-micron tolerances. They created a custom jig to hold the A1 sensor-motion sub-assembly precisely in place on one of these machines so they could tweak its alignment precisely as needed.

 

Here's the Canon EF-mount adapter on the front of the A1 sensor unit, with the lens removed.

It takes about 40 seconds to fully test an APS-C lens on the production line, about 60-65 seconds for a full-frame one, given the need to move the sensor to cover the full 35mm frame. (I'm a little surprised that there's not more of a time penalty associated with moving the sensor for full-frame lenses.)

 

A closer look at a Nikon F-mount adapter. These adapters are made from stainless steel, with a very robust threaded mount to connect them to the measurement head. The lens adapters are made to very exacting standards, and the underlying sensor sled is very precisely aligned with the mount, using coordinate-measurement systems similar to the one I snapped a picture of during my previous tour of Sigma's Aizu factory.

The A1 system obviously needs to be able to accommodate multiple lens mounts (including all the electrical signals that pass between the camera and lens, to control focal distance, aperture, etc.). The solution was to build the A1 housing with a custom mounting flange with a very short back-focus distance (also called flange focal distance; basically the distance from the front face of the mounting flange to the image sensor surface), and then create adapters to provide both the correct mechanical and electrical connections for each lens mount. The shot above shows an adapter (for the Nikon F-mount) in Ishii-san's hand. The second image above shows the mount attached to the A1 system itself. The ribbon cable carries the electrical signals down to the A1's controller, letting the computer "talk" to the lens.

This shot shows a partial view of the A1 system's output screen. What's missing (for obvious proprietary reasons) on the right side of the screen is a display of MTF values vs the various pass/fail thresholds for the lens under test. The system has two modes it can operate in; a production mode, suited to rapid testing of lenses coming off the production line, and an "R&D" mode that allows for a greater variety of tests, at arbitrary locations within the frame.

The role of the A1 in design process

Noting the "R&D" mode that it offers, I asked what role the A1 system has in the design process. Can it perhaps show problems that weren't evident in their lens-design software? That is, are there things that Sigma's ray-tracing software doesn't model well, but rather have to be evaluated in terms of the actual performance of physical samples?

Interestingly, the answer was that the issue wasn't with the ability of the design software to simulate lens performance correctly, but rather the extent to which the factory processes could produce physical lenses that matched the design specs carefully enough. In other words, it wasn't a matter of verifying the accuracy of the ray-tracing software, but rather one of evaluating the ability of the factory's processes to faithfully execute the design in glass, metal, and plastic.

 

Sigma's CEO Kazuto Yamaki views Sigma's factory in Aizuwakamatsu, Japan as a key part of their success. He and his father made a strategic decision a number of years ago, to keep their entire supply chain based in Japan. This means higher production costs in some areas, but helps them maintain an exceptional skill level among their factory workers, and excellent quality control from their suppliers. Mr. Yamaki told me that most people who come to work for Sigma never leave for other employers. He views the workers' skills as critical for producing high-spec optics like their Global Vision line of lenses.

Manufacturability and Manufacturing Ability; the keys to great optics

This points to what might, in fact, be one of the most important things I learned on this visit: Sigma works very hard to maintain an especially close coupling between their lens designers and factory operations. One example of this is that the creator of the A1 system Mr. Ishii had previously been a lens designer himself. After creating the A1 system, he moved to Aizu to act as a liaison between the factory and the engineering staff at headquarters.

Sigma's CEO Kazuto Yamaki told me that he's constantly pushing the factory to be able to come ever closer to the theoretical lens performance predicted by the design software. Whenever the A1 system uncovers a shortcoming, it presents an opportunity to more finely tune their production processes.

Here's a brief exchange I had with Mr. Yamaki on the topic, focused on Mr. Ishii's role:

Dave Etchells: So Ishii-san doesn't just do A1 development and production, he's also involved in bringing lenses to manufacture?

Kazuto Yamaki: Yes. Actually, when he was in charge of the A1 project, he worked in the head office [in Tokyo], as an optical designer. Once we started using the A1 in production, I asked him to move to Aizu and help the production engineering team, because it's so important to have a very high production capability. Otherwise, even if we design a lens very nicely, but the production quality is low, we have to lower the criteria. [The QC standards for accepting or rejecting lenses] What I want to do is to put the criteria very close to the design data. [That is, he wants to set the QC standards very close to the theoretical ideal, as determined by the ray-tracing software.] This is really challenging, but otherwise we cannot make a lens that works for very high resolution cameras. Like the 8K camera, it requires a very, very high performance.

[I found this an interesting note about 8K cameras, as that's more of a video or cine term; 8K would refer to a camera with roughly an 8,000 x 4,500 pixel image (36 megapixels) in the HD 16:9 aspect ratio or 8,000 x 6,000 pixels (48 megapixels) in the standard 3:2 aspect ratio found in most DSLRs.]

DE: So the way things work is that the designers say, 'Here's the design,' and there's a production engineering team here that Ishii san is part of that decides, 'Can we make it? How can we make it match the design criteria?'

KY: Yes, we make the criteria while we are developing lens. We measure each prototype in pre-production, and we know how high the criteria…

DE: How tight you can set them.

KY: Yeah.

DE: In a previous conversation, I recall you talking about factories sometimes rejecting designs as unmanufacturable. But you said that it's very rarely that Sigma's factory will reject a design. That never really occurred to me, before you mentioned it. So sometimes it will happen that engineers will design something, but then the factory looks at it and says, 'We can't manufacture this. There's no way to build it.'

KY: So he also plays that role. Once the optical engineer in headquarters comes closer to the completion of the design, they submit the data to him. So he checks if there would be bad yield. The optical designer can simulate the yield, but also Ishii-san also checks the lens construction and the sensitivity of each of the lenses, and he judges whether the factory can accept it or not.

DE: So part of what he does then is a sensitivity analysis.

KY: Yes, sensitivity analysis, yes. 

[Sensitivity analysis involves looking at each component of a design and asking what would happen if it were at one side or the other of the tolerance range that can be held. It's a very laborious process: Because you have to look at all the possible combinations of each component being at one side or the other, the number of variations to be evaluated expands rapidly. If there are two parameters, there are four combinations of either being at one side or the other of the allowable range. If there are three parameters, there are eight combinations. By the time you get to 10 parameters, there are 1,024 possible situations. Now, consider a lens with (conservatively) a dozen elements, and that the parameters include not only tolerances on the lens shape, but x, y, and z-position and rotation about either of two axes of each element as well, and you can begin to appreciate the complexity involved. It takes a deep understanding of both optical design and manufacturing processes to arrive at meaningful conclusions.]

DE: And knowing the production process, he can say 'Well, I know in production this element might shift like this.' Then he can run that through a simulator and see if that will affect yield or not.

KY: Yes. But if the factory rejects too much, we cannot make very high performance lenses, or we would have to just make a very boring normal lens. In this case, he starts a new project, to strengthen our ability to manufacture, or a new method for optical alignment.

DE: So it may be that a design comes in, but it gets rejected; 'We can't make that.' But then Ishii-san will say, 'Okay, now the reason we can't make that was X, Y, Z.' Then he figures out a way for that not to be a problem anymore.

KY: Right.

DE: It sounds like a very challenging job!

KY: Yes. So it's not a joke, that having a good relationship between the designers and the people in the factory is really important. If the people in the factory hate the designer, they reject anything anyway. 

DE: Yeah. You don't want it to be like, 'Oh, here's that guy again.'

KY: That could put the company in a very difficult position. So having a good relationship between the designers in headquarters and the factory engineering designers, the manufacturing engineers, is really important.

DE: So it's kind of like Ishii-san is sort of the face of the factory to the designers, then.

KY: Right now, yes. In particular, he is in charge of the optical manufacturing design, especially for optical performance. His colleagues, some of his other colleagues are specialists for mechanical things.

Summary
Once known primarily as a maker of lenses for the budget and entry-level end of the market, Sigma has in recent years emerged as a true world leader in high-quality optics. Lenses like the Sigma 50mm f/1.4 Art and 18-35mm f/1.8-Their internally-developed A1 lens testing system and the 100% computer-based QC testing it provides for their Global Vision lens line has been a key component of their success. Sample variation has always been a bugaboo with really high-performance lenses, but the 100% digital QC testing of Sigma's Global Vision lenses provides a level of assurance to end-users that I believe is unmatched in the industry.

I greatly appreciate Mr. Yamaki allowing me to share the story of the A1 system with our readers, and particularly his permission to show detailed photos of such a highly proprietary part of Sigma's operations. 

(Other manufacturers: We have a proven track record of careful handling of proprietary information of this sort - wouldn't you like to have the attention and mind share of our readers for your manufacturing processes? :-)