Sony A6400 Field Test Part I

Sony A6400 Field Test Part I

Amazing achievement, abject failure or just a pretty good camera?

by Jaron Schneider & Dave Etchells | Posted 02/11/2019

We had the opportunity to shoot with the new Sony A6400 at an event Sony put on in San Diego a couple of weeks ago. It's taken us until now, though, to digest our thoughts and experiences with it.

Sony's new subject-tracking technology is pretty incredible, particularly on the A9, based on a firmware update that will be coming in April 2019 (Read our article about Sony's amazing Real-Time Tracking AF). Our experience on the A6400 was quite a bit different though. On the one hand, it fell drastically short of what we'd been led to expect, and relative to the implementation on the A9. Does that mean the A6400 is a failure, though? That's the difficult question we had to answer, and that's taken us this long of inspecting literally thousands of shots we took with the camera to sort out. Read on to see where we stand on the camera currently. (As we write this, we're still waiting for a production sample to put through more rigorous tests in our lab and the real world; we're told one will be forthcoming soon.)

The Sony A6400 is definitely not "entry-level"

The Sony A6400 is the latest iteration in the A6xxx product line, and Sony says it will become their new "entry level" model, with the A5100 going away entirely. The A6000 will apparently continue to be sold for a while yet, but Sony was clear that going forward, the A6400 will represent the bottom end of their APS-C line.

The terminology puzzles us, though, because neither the price nor its features are remotely entry-level. Relative to their current lineup, the A6400 is listed for $900 body-only or $1,000 for the kit with a 16-50mm zoom lens. By comparison, the A6000 is currently selling for only $390 body-only, or $648 with a 16-50mm zoom. That's a huge gap between the two!! Comparing to other brands, $1,000 is way beyond an entry-level price, with cameras like the Nikon D3500 and Canon T6 both selling for just $400 for kit versions(!)

All we can figure is that Sony plans to bring down the price of the A6400 dramatically over time, but for now it should be viewed as much more of a midrange model, competing both on price and features more with models like the Canon 80D, Nikon D7500 and Panasonic GX9 than either company's lower-end cameras.

Confused expectations

The A6400 is a complicated camera to evaluate, not just because it wasn't the camera the rumor mill had promised to avid Sony fans (and therefore was somehow a disappointment for not living up to the manufactured /fake hype), but also because it was shown with a brand new "AI" autofocusing feature side-by-side with the same feature on the top-of-the-line A9 (coming via future firmware).

Sony can hardly be held responsible for unreasonable expectations set by the rumor mill, but the language and presentation videos they used to describe the A6400's new AF tracking system seemed to place the small, "entry-level" $900 camera in the same league as the much larger and more robust $4,500 A9. Watching the presentation, we were left with the impression that the new camera would have similar tracking performance to the A9, and it could even have been construed that the A6400 was better than the A9 when it came to subject acquisition speed, because we were told that the A6400 has "the world's fastest autofocus acquisition." This ability was said to be thanks to "a new generation BIONZ X processor borrowed from the Alpha 9." The hype all added up to amazing performance in the AF world.

After using the camera and speaking to engineers about its design and purpose, the reality just didn't measure up to the initial billing. In fact, when asked after the fact whether the A6400 and the A9 did indeed share the same processor, the official Sony answer was, "we cannot say." So it seems that the statement in the rollout presentation that the processor was "borrowed from" the A9 was more in the sense of "yeah, the general architecture is the same" than "it's the same chip."

For those unaware of how the different parts of a camera come together to create a whole system, saying that the camera shares a processor with the A9 and stating that it's autofocus acquisition speed is unrivaled by any other camera ever made, makes it sound like its performance is equal. But the processor is only one part of the A9's equation. It has a whole lot more going on, including its dual layer BSI sensor, it s markedly faster sensor readout speed, and its memory layer on the sensor itself. Even if the A6400 had exactly the same processor as the A9 (which as mentioned was not even verified by Sony), that's not enough to make the camera actually perform like the A9. But it sets that expectation with users, and it's a great way to sell a camera.

That expectation of the highest-end level of performance was set before we even touched the camera. And that's a shame, because if were we to use the A6400 in the right context and without the marketing posturing placed in front of us prior to handling the camera, we would have come into our testing with a much more leveled view. This is a $900 "entry level" camera (although as noted above, that price bracket isn't quite "entry-level"), and is the lowest-end mirrorless that Sony will make going forward. Our expectations should be in that zone, but the presentation and simultaneous rollout with the A9 firmware upgrade had us comparing it to the A9, and justifiably so.

Sony A6400 Real-Time Tracking: Stills

The A6400 has a lot of features worth mentioning, but the number one most-hyped feature is its autofocus and the latest technological breakthrough in that arena from Sony: Real-Time Tracking.

Real-Time Tracking is Sony's first attempt at autofocus recognition that goes beyond just eyes or faces and includes patterns, textures, distances and shapes. Rather than having to rely on just recognizing eyes or faces, Sony's new technology is able to identify whole objects and, in theory, track them indefinitely across multiple planes of focus and despite interruptions within the frame (that is, other objects passing in front of the tracked subject). Though strong on its own, the technology is vastly superior when combined with Eye-AF, and in stills-shooting the two features cannot be separated. Starting with the A6400, Eye-AF combines with Real-Time Tracking to continuously calculate the optimum autofocus point, whether or not an eye is in view. Now, the side or back of a subject's head no longer interrupts Eye-AF, and through a full spinning motion (like a basketball, football or ice skating action, for example), a subject can be perfectly tracked and no shots will be out of focus.

Sony calls this "AI" technology, but we'd stop short of going that far. The concept certainly does a more complete job of evaluating a scene and a subject than we've seen in most previous cameras, but it's still short of being able to think for itself, learn and get more powerful through use, or even implement the kind of "deep learning" object recognition we saw recently in Olympus' new E-M1X. In short, using the term"AI" in this instance seems a bit deceiving.

So how well does the tracking work? Our short answer is, it depends. Let's start with the best possible scenario and move backwards from there.

According to the Sony engineers, the best situation for Real-Time Tracking is with a high shutter speed and lots of light. This makes sense: the more light, the better the result, as this is generally the case for photography overall. So our test in this scenario involved following a volleyball player outside in the late afternoon, shooting at high shutter speeds and low ISO.

We went over all the volleyball shots in one particular sequence with a fine-toothed comb and rated the focus from 0 (truly out of focus) to 1.0 (fully tack-sharp on the face/eyes), and counted any that we rated 0.7 or higher as being keepers. On that basis, the keeper rate was about 75%. If we relaxed our standards a bit, the keeper rate was 82%. That result is absolutely not bad for an entry-level model, especially considering it was shooting 11 frames per second. That's a lot of photos for a beginner or amateur shooter to end up with, and keeping 80% of them is pretty darned good.

But Sony set the expectation in their presentation that the A6400 would compete with the A9, a camera that typically gives 95% or more keepers per the same number of images. It's no surprise that a camera costing only 1/5th the cost of the A9 would perform at a lower level, but our expectations were set a lot higher going in.

During shooting we noticed something, though: the camera seemed to be giving us conflicting information. While looking through the viewfinder, we noticed that when we were just tracking a subject, half pressing the shutter button or using a custom back button to activate tracking, everything would look great. Beyond great, in fact. The green autofocus indicator had very little trouble perfectly following a subject.

But the second we started taking photos, that green autofocus indicator would wander all over the frame relative to the subject's motion. When looking through the viewfinder, it appeared that the autofocus was not properly tracking.

When we went back and looked at our images, though, we saw that in many cases the camera was in fact still tracking the subject perfectly well. So... what was happening? It seemed that the camera wasn't able to process tracking, capture, image storage and proper display all at the same time. It seemed to be too much of a workload for the small camera, so the engineers (rightly) prioritized focus and image acquisition, only updating the display when there was time, once the most critical operations had been taken care of.

We decided to record what we were seeing in the camera during image capture and compare that to what the camera was writing to the memory card, to see if there was some kind of lag.

In hindsight, we realized that we really needed to have had a second camera that could record both the viewfinder display and the live action at the same time, to properly show the lag. We couldn't determine after the fact just which images were being recorded at any given point in our video recording of the viewfinder display. (This is something hope to test properly, whenever we can get an evaluation sample at IRHQ to work with.) The video does show accurately, though, the way the AF point would wander from the main subject, and its sometime preference for the net as the AF target.

Subjectively, it seemed like the viewfinder image followed the subject fairly well (we didn't feel like there was a huge amount of viewfinder lag), but the displayed AF tracking box was often way behind where the subject actually was in the frame. Based on what we were seeing in the viewfinder, it seemed like the A6400's Real-Time Tracking was failing almost completely. As noted above, though, the actual focus tracking in bright-light situations was better than what was displayed in the viewfinder.

This problem was, as expected, most noticeable when shooting at the maximum frames per second in High+ drive mode. In lower speed continuous modes, the problem became less noticeable, as the camera had to process much less information. At the lowest continuous shooting speed (about 3 frames/second), the problem went away entirely, but 3fps is a pretty slow continuous shooting rate these days.

When shooting in darker or dim situations, however, it's clear this technology has lot of room to improve. Shooting again at high burst speeds, I captured a 23-image sequence of a man juggling bowling pins. In that image sequence, using the same parameters for deciding if an image was in focus or not as before, the keep rate plummeted to 43%. Over half the images captured were either completely out of focus (as if the AF on the A6400 totally hunted off subject for a couple frames) or mostly out of focus.

Under those conditions, it seemed that the camera was smart enough to know where the subject was, but somehow lacked the capability to capture those subjects cleanly a majority of the time.

These were admittedly somewhat low-light conditions (but they were the ones Sony presented us with to experience the camera's AF prowess), but even for an entry level camera like the A6400, that is a very low keeper rate. I'm glad that it was able to capture any images crisply at all, but I have a hard time recommending the A6400 if you plan to try and capture action in a dimly lit environment on a regular basis. Again, this was a lighting scenario they chose themselves, and the subject was in clear view.

Sony A6400 Real-Time Tracking: Video

When we spoke to engineers regarding the performance of the A6400 in low light, they informed us that it would certainly perform better in bright light (as we confirmed), but also let us know that specifically with the A6400, AF tracking during video capture should be a superior experience to stills capture.

Unlike regular Eye-AF, which is not available on any Sony camera during video capture, Real-Time Tracking is able to be harnessed while capturing even 4K video on the A6400. (As a note, when asked why Eye-AF was not able to be used in video, the response was one we've come to expect from most manufacturers when asking about technical details: "We cannot say.") Real-Time Tracking in video sounded outstanding. The idea of being able to capture a moving subject and track them as they move around a scene is normally quite the challenge, especially with a telephoto lens and if their distance from the camera varies wildly in any particular scene.

We decided to test it in a couple different scenarios. In one, I went back to the juggler and tracked his face while he was stationary. As I zoomed in and out, the tracking did waver slightly as it attempted to keep track of the subject, but it did a very respectable job and I wouldn't say that it ever truly lost focus. It dipped in and out of perfect, but it stayed on task.

In the second test, I had my friend Micah from NoFilmSchool just walk towards me with a mixed lighting background. In the video, the tracking holds on Micah for a bit before freezing up and holding steady on the background. That was a poor result.

Additionally, I had Micah start on my right and move from my right to my left and walk behind someone else. The AF tracked him right up until the moment he walked behind another person, and the focus then jumped to that person and lost track of Micah altogether. This was another poor result, albeit in the same dim indoor lighting mentioned earlier.

Remembering what the Sony engineers told me about light, we tried more autofocus testing in video outside at a basketball game. The result was... mixed. If a subject moved in a straightforward manner or did not go behind or around any other people, the camera tracked just fine. However, once he made a particularly erratic motion (common in basketball), focus would dip off and I would have to manually reacquire. For what it's worth, it seemed that if the subject passed behind another object and there was a sufficiently large difference in distance between them, the camera did a better job of not jumping to the foreground object. We weren't able to determine just where the threshold in terms of difference between subject and intervening-object distance lay, but a very rough guess would be that if the interfering object was less than 60% of the distance to the subject itself, the camera would track fine. If it was more than 80% of the main-subject distance, it would jump nearly every time.

In the final test, I tracked volleyball players in very bright light. In this case, even with a net between me and the subjects, the autofocus was far more reliable, though still not 100% perfect. There were plenty of instances where the Real-Time Tracking accurately captures the players for nice, long sequences. But there are others where the focus would get distracted on a background or foreground element, specifically the net.

Real-Time Tracking: Final Notes

Because we were able to test the A6400 as well as the A9 with beta firmware that also implemented Real-Time Tracking, I am aware of what the technology can do. The A9 was a wholly different experience than the A6400: it's much, much better and more consistent. So knowing that the technology is sound helps me formulate opinions on it as implemented on the A6400.

In stills, it's slightly more powerful in straight-up tracking because it combines with Eye-AF. When the camera is able to mix the power of Eye-AF with Real-Time Tracking, the result is a very well tracked subject.

But that's just tracking.

When it comes to actual capture. It is a mixed bag. The camera is far from being an A9 and doesn't seem to have the horsepower to process all the data it is being asked to deal with. The result is an autofocus experience that sometimes misses a moving subject in bright light, mostly misses moving subjects in low light, and provides an inaccurate representation of reality to the shooter due to some pretty bad lag in the viewfinder.

All that said, though, as an "entry-level" camera, it's still pretty good. (Just without an entry-level pricetag.)

In video, the lack of Eye-AF assisting the Real-Time Tracking is more evident. Though Eye-AF isn't necessary for Real-Time Tracking to do its best (we saw excellent results from the A9, so we know what is possible), the A6400 has difficulty locking on to subjects when there are foreground or background elements that could distract.

In bright light such as a volleyball game, results improved. Just be aware that any foreground elements might distract the focus point, and you may lose your subject. It did better here than anywhere else, but it was still far from infallible.

In a basketball game, it's hard to recommend the tracking in video. There are too many similar shapes and depths that the camera cannot differentiate them. It would often jump to different subjects, jump to the background, or lose track of any subject altogether. Though it's possible with a better camera system like the A9 to use Real-Time Tracking without the benefit of Eye-AF, the A6400 just feels underpowered to do so properly.

I wanted to make note of two other bits of information regarding this feature:

1) Sony engineers showed us examples of Real-Time Tracking working well in video in a demonstration they put on for us the day after we shot with the camera for the first time. They pointed out how well it tracks, and noted that higher shutter speeds resulted in better tracking performance. During this presentation, one of the journalists in the room asked why they were shooting at such high shutter speeds (over 1/200 second) when demonstrating video. If you shoot video, you get why he asked this. No videographer who knows what he or she is doing would ever shoot at that high of a shutter speed outside of slow motion. It results in bad, jittery-looking footage. The Sony engineers didn't specifically answer this question, and it seemed that this was a factor they had not fully considered.

2) The only way to activate Real-Time Tracking while in video mode is to use the touch screen. By default, touch is either limited in function or off altogether, and then you also have to tell the camera to enable touch during autofocus. Both those toggles are located in wildly different locations in the menu, which makes them hard to find and activate. To those who don't read the instructions or are not told ahead of time, attempts to use Real-Time Tracking either by the menu option or via a custom button will be greeted with an error message that inaccurately diagnoses the problem as your shooting mode. If you are in Manual mode, it will say you cannot use the feature because you're in Manual mode. If you're in Program, it will say it's because you're in Program. Same with Shutter Priority and Aperture Priority. If you were to just go on what the camera was saying, it's effectively stating that you cannot use Real-Time Tracking because the camera is a camera and it's in a camera mode. That was frustrating.

Additionally, the touch screen isn't the best user experience for selecting a subject. It's hard to pinpoint exactly the person or object you want to track, especially if they are moving. It felt clumsy and inaccurate. Overall, it was not the best user experience. I'm not sure why they removed the ability to use buttons while shooting video and trying to use Real-Time Tracking, but doing so resulted in a lackluster shooting experience.

Bottom line: Who's it for?

If you're a little confused by the above, welcome to the club; honestly, we need more time with the camera back at IRHQ to fully understand how it stacks up against the competition. At its core, though, it's a very capable camera, and for single-shot stills, its tracking AF works extremely well in bright lighting conditions. Likewise, its video AF is sure-footed and smooth, as long as it's not being asked to deal with rapid motion or objects passing closely in front of your subject. So here's a quick breakdown of how we think the A6400 will work out for different groups:

Vloggers

This is a major market Sony seems to be aiming this camera at. And it can be a great fit here, with great video quality and accurate, smooth autofocus. (A perfect example would be a vlogger talking about a product, with the camera focusing on the product when s/he holds it up to the lens, and then smoothly transitioning back to their face when the product is lowered. The rear panel LCD is either great or awful for watching yourself, depending on whether you're using a hot-shoe mounted mic or not. Mounting a mic on the shoe blocks the LCD, making it useless for watching yourself. Use an external mic and you'll be golden.

Family picture- and video-taking

The A6400 would be a great camera to get, to grow up with a new baby. The Real-Time Tracking feature would work great for any kind of single-shot usage, even with a highly active toddler. (Half-press the shutter button to activate the tracking, fully-press when you want to snap the picture.) Autofocus works pretty well with moving subjects as well, in both still and video modes, but you'll need to pay attention to tracking your subject manually as well, not letting their position within the frame change too quickly. It won't be a good choice if you're trying to pick your kid out from a crowd of others running back and forth, though.

Not for sports

Pretty much forget about it, if you're dealing with rapid action and multiple players that you want to capture with either continuous shooting or in video. If the subject moves around the frame too quickly, or if other players cross between you and your subject, with not a lot of distance between the two, focus is going to be spotty, jump to other players or to the background. Depending on the sport, and given the A6400's fairly high continuous-shooting rates, you might end up with enough keepers to satisfy you. You could also get pretty frustrated though.

Landscape, still life, portraits, etc.

No problem here, but then you don't need an advanced AF system for these subjects anyway. You'd likely be quite happy with the much cheaper and more truly entry-level A6000.

There is a lot more to discuss

As you probably noticed, we only discussed one feature on the A6400 in this field test, and spent over 4000 words doing so. The reason being that 1) this was the most hyped feature of the camera from Sony marketing and 2) there was a lot to dissect about it.

There has been a lot of talk regarding the AF performance of the A6400, and we hope we added to that conversation with this breakdown of just Real-Time Tracking. We have so much more to discuss, like the build, image quality, ISO performance and video quality. Stay tuned for more on that as our review continues, particularly now that a production sample has finally arrived in our lab for testing.