How to Read our Image Stabilization Test Results

There's no commonly-accepted way of describing the detailed behavior of image stabilization (IS) systems, so we at Imaging Resource have had to come up with our own. We have a more technical article here which explains in greater detail how we go about measuring and analyzing IS performance; but in this article, we just want to describe how to read and interpret the graphs we use to present our findings, and how to translate those graphs into real-world results. (If you have questions not answered here or in the Image Stabilization Testing White Paper, we invite you to ask them in the forum thread below.)

IS performance is such a complex phenomena that there's really no practical way to characterize system performance for all possible combinations of camera, lens, shooter, and shutter speed. What we do instead is try to measure and describe performance for conditions that should simulate the experience of most of our readers. That is, we test system performance when being used by a rather steady and a rather shaky shooter, and in the case of zoom lenses or cameras with body-based stabilization, we test at both shorter and longer focal lengths.

Top-Level Summary

We realize that many readers just want the basic numbers, so we make those available at the very top of each IS review page, in the form of tables showing the performance our "steady" and "shaky" testers measured. Here's an example:

Because the performance of IS systems can vary quite a bit depending on how steadily a person can hold the camera, we always show results for both "steady" and "shaky" shooters.

The IS Off and IS On numbers in these tables show the slowest shutter speed that each tester could shoot at, while getting an average shake-produced blur of 0.5 blur units. (See below for what a "blur unit" looks like.) The difference between the lowest usable shutter speeds with IS On versus IS Off is the improvement, which we translate into stops of improvement in the last column.

You can get an idea of what you might be able to expect from an IS system by comparing the slowest speed you can hand-hold a lens of the same focal length with the slowest speeds each of our testers was able to use with the IS turned off. If your hand-holding experience matches that of our Shaky tester, then your results with the IS stem in question will probably parallel his. Likewise if your performance matches our Steady tester.

For reference, the 1/FL column shows what the "1 divided by the focal length" rule of thumb shutter speed is for the lens and camera in question. (Calculated based on the effective focal length of the lens on the body it was being tested with.) In the example above, our "shaky" shooter's IS-off results followed the 1/FL rule pretty closely, while our "steady" shooter turned in a performance that was quite a bit better than that benchmark. As seems to frequently be the case, the IS system in the lens therefore didn't help the Steady shooter quite as much, but he was nonetheless able to shoot at dramatically lower shutter speeds with the IS enabled than he was without its help.

These tables will probably be all that 90 percent of our readers want to know about our IS test results: They give a good idea of what to expect from a system, based on how steadily you can hold a camera. We do have many readers who want to know more of the technical details, though, so we've prepared more in-depth graphs for them. If you're part of the 90 percent, you can stop reading here and go back to the IS review you were just reading. For the remaining 10 percent, read on below.

Summary Graph

For those wanting a little more detail on how an IS system tested-out, the graphs you'll see in the reviews reveal more of our findings, including data points for each individual shot captured during our testing.

This graph is the one we use to summarize our test results over the photographically relevant range of shake-induced blur. The two curves show the average increase in blur that resulted from camera shake, plotted as a function of shutter speed. (Note that this isn't lens sharpness, this is just the increase in blur due to camera shake.) The red curve shows the blur we found with the IS turned off, the blue curve shows the blur with the IS system turned on. The dashed vertical line marks the shutter speed that corresponds to 1 divided by the effective focal length of the lens being tested, a common rule of thumb for the slowest shutter speed that can be routinely hand-held. The two green arrows show how much the IS system improved the blur results, depending on how much blur you can tolerate for your application. The upper blur threshold of 1 blur unit would apply if you're only making relatively small prints from your pictures, perhaps a maximum of 5x7 inches or so. The lower threshold of 0.5 blur unit is a more generally-acceptable standard, resulting in images useful for making larger output. When comparing systems to each other, we use the more stringent criteria, requiring that shake-induced blur be less than 0.5 blur unit.

Details: How Many Shots were Sharp or Blurry?

It's important to note that these graphs show average blur performance. Shooting at moderately slow shutter speeds, you'll generally find that some shots are blurry, while others are relatively sharp. What IS does is shift this balance in your favor, resulting in a better percentage of sharp shots than you'd get on your own. We experienced this in our testing, and to help you know what to expect in your own shooting, we show you how many of our own test shots were sharp or blurry. If you roll your cursor over the summary graphs in our reviews, you'll see a number of points appear, against a colored background. The image below shows the colorized version of the graph above, without the rollover, to make it easier to examine as you read this section.

Each point on this colored graph represents a single test shot: The left/right position shows the shutter speed the image was shot at, and the height shows how sharp or soft the test image was. Low dots correspond to sharp images, high dots to blurry ones. To help you interpret the results, the shaded background shows levels of blur that are generally acceptable (green), marginal (yellow) or unacceptable (red). (Note for the data geeks: This view offsets the data points slightly from the plotted curves, to simplify the presentation of IS improvement. Read our Image Stabilization White Paper for an explanation of this.)

This view makes clear how much variation there is from shot to shot. In the example shown above, our tester managed to get some very sharp images with the IS system turned off, even at a shutter speed as slow as 1/60 second. (The red dot that's almost on the baseline, just before the 1/50 second mark.) At the same time, even with the IS system engaged, he still got one or two relatively soft shots at shutter speeds as high as 1/120 second. (The blue dots that just edge into the yellow area, just to the left of the 1/100 second shot.) It's easy to see, though, that the IS system was a considerable help, in that the majority of shots from 1/200 down to about 1/30 second were acceptably sharp when it was engaged, vs. only two or three sharp shots in that range when the IS system was turned off.

More Gory Details: The "All Shots" Graph

The graphs just shown display our results with IS On and IS Off across a range of sharpness that's photographically useful. For those readers who want to see all the data we captured, we provide links to larger graphs, that show the full range of our experimental results.

In this graph, there's no offset between the plotted curves and the data points, and the data values extend to much higher levels of blur. The data at high blur values isn't too relevant to day to day photography, since the images there are so blurred. In the interest of full disclosure, though, we provide links to these graphs for people who may be interested in how the IS systems handle very long exposure times. In our IS reviews, we offer these graphs as links only, so they won't clutter the pages for the average reader.

Still want to know more?

If the results tables alone met the needs of 90 percent of our readers, the tables plus the graphs just described are all that probably 99 percent of people care to know. For the remaining one percent, we offer our Image Stabilization Testing White Paper, which explains our IS testing methods and data analysis techniques in full detail.