Aided by automation, Lensrentals developed cutting-edge rapid MTF test for photo & video lenses

 

 

Lensrentals founder Roger Cicala has been testing lenses for nearly a decade and his methodology has continually improved not only in efficiency, but more importantly, in accuracy. There are numerous ways to test lenses and there are advantages and disadvantages to different methods. More accuracy typically comes at the cost of additional time and money, which are not infinitely-available resources. Further exacerbating the challenges of improving optical testing is that cameras are becoming higher resolution, which can render older testing methods and equipment obsolete, unable to catch lens problems today's best cameras can resolve and expose.

In pursuit of testing which will not only prove more accurate, but also much faster, Roger Cicala and his team have spent the last year developing a Rapid MTF Test for photo and video lenses. To illustrate its importance, let's consider some of the other optical tests they've performed. It started with computer analysis of chart images, which worked okay, but was slow and the results weren't consistent enough. An MTF bench -- MTF stands for Modulation Transfer Function, and you can read more about it here -- was effective in certain measurements, but it missed issues with rotational asymmetry in lenses. For example, a lens could excel in on an MTF benchmark test but struggle in real-world use as perhaps one edge or another was noticeable worse. To improve these tests, the team started doing 4-rotation MTF tests, which exposed issues with each quarter of the lens. 8 and 12-rotation tests proved not to offer additional information for the considerable increase in required testing time.

 

With Lensrentals' four-rotational tests, they were able to better evaluate whether a lens had issues with one side or another. Image credit: Lensrentals

Armed with four rotational MTF charts, you get a lot of information. However, the information was not that easy to digest. The team improved the plotting of the test results, helping make the information intuitive to view. Now, Lensrentals had rigorous testing and a good way to display the information. Another issue arose, when a lens didn't look very good on the test charts, the methods weren't necessarily explaining what the underlying problem was. This highlighted an important consideration. When you use a lens to take photos, you are very rarely capturing images of a two-dimensional subject. Rather, you are capturing something within a 3-D scene. Some lenses came up short when shooting in three dimensions despite excelling in a rotational MTF test.

Enter field curvature testing. This showed how a lens resolved in three dimensions rather than two dimensions and did a much better job showing if a lens was sharp on all sides or if there was tilt-caused softness. Armed with even more information, the Lensrentals' testing team now could tell if a lens was sharp at four rotations and sharp in three dimensions. Importantly, if a lens struggled somewhere along the line, they had the necessary information to identify the optical issue. Ultimately, the testing was so precise and rigorous that they published information in peer-reviewed journals and the methods were used by some manufacturers and end-users such as Caltech/NASA Jet Propulsion Laboratory. High praise, to be sure.

 

"We start with the simple test that is the basis for MTF vs. Field vs. Focus; a test called the Through-focus MTF. Let’s look at another MTFvFvF graph. The horizontal axis is the lens field from one side to the other, and I’ve drawn a black line at the center of the lens’ field. The vertical axis is focusing position and “0”, in the middle of the vertical axis, is the proper focusing distance for the lens we’re testing." Image credit and text: Lensrentals

What's the problem, then? It takes an hour or more to test a single lens like this and Lensrentals needs to test hundreds of lenses a day. Further, cameras continually received higher-megapixel, higher-resolution sensors, continually changing the standards by which lenses ought to be tested. A lens may do very well on a 24-megapixel camera but struggle on a camera nearing 50 megapixels. They needed faster and better. And Cicala, after a year of hard work, has come up with a computer-assisted method, which you can learn more about at Lensrentals. 

The gist of the new method is that the team now has a computerized optical bench and test target which evaluates lenses at their center and their corners. Each lens has around 200 or more customized scripts to run the automated testing system. Every lens needs its own custom scripts, including different scripts for the same lens available in multiple lens mounts. Once a lens has been established into the testing, the team can evaluate it against many samples of the same lens and the computer quickly gives it a passing or failing mark. There is a 0.5 percent error rate, but this error rate only gives a failing mark to a lens that should have passed, meaning that there aren't poor lenses being sent to consumers, but rather only good lenses being held back for additional testing.

 

This shows the results using the new Rapid MTF Test of four copies of the same wide-aperture lens. The bottom right graph shows a lens which failed the test, whereas the top two lenses are perfect. The bottom left sample is not as good as the top two, but still passed the test. Image credit: Lensrentals

There are still some limitations and challenges, but the team has worked hard and proven, contrary to what some in the industry believed, that the new rapid MTF testing methodology can not only work but work incredibly well and with very good accuracy. Anyone even semi-interested in optical testing should do themselves a favor and read Roger's article. 

(Via Lensrentals)