Detecting Photoshopped Images: Part Art, Part Science

On May 6th of this year, the Christian Orthodox Patriarchy of Moscow published a photo of the church’s Patriarch Father Kirill sitting at his desk studiously looking at some papers. Within hours, this image had stirred up a firestorm of controversy when it was noticed that in the polished desk’s reflection of the patriarch’s wrist there is a wristwatch that does appear on his actual wrist (see red arrow.) The wristwatch in question, identified from its reflection, was a $10,000 Breguet Swiss watch.
 
In a snap, the Moscow Patriarchy was caught red handed trying to make their leader fit his public relations image of as a humble, nay, a "poor" Patriarch.
 
Distorting and manipulating images like this with Photoshop and other editing software has become commonplace. Fashion magazines do it all the time and it has become a major problem for editors, police and the rest of us.
 
Kevin Connor worked at Adobe for 16 years in product management before he left to join forces with Dr. Hany Farid, to develop software to help detect "Photoshopped" images.
 
Through their startup company Fourandsix they hope to create tools to help police, editors and others, identify altered images. They will roll out their first product later this year.
 
“There’s a temptation to want to have some magic bullet or magic algorithm that will tell you whether an image is real or not, and we quickly realized that’s just not going to work,” Connor says. “What you have to do is approach it as a detective and examine all the various clues in the image itself and the file that contains the image.”
 
Naturally, bad Photoshopping abounds and it is generally easy to spot, as the examples in this story prove. But whether it is a movie star's breasts altered for the cover of a fanzine or an out of favor politician getting erased from a reviewing stand, these manipulations are getting more sophisticated and harder to detect.  
 
Three Ways to Detect Photoshopped Images
The solution, Connor says, requires “not one but a series of technologies” and a trained person making an informed call. He suggests three strategies for detecting Photoshopped images.
 
His first is to check the metadata. All digital photo files contains metadata, some of it as EXIF data, which you can read by a tool such as this one or this Firefox add-on. The EXIF data records things like what camera took the image, what aperture was used, the ISO and so on. Often it records the last piece of software used to save the image, but other factors can distort this information.

 
“When you save a JPEG there are choices that the camera or software has to make about how to store that JPEG,” he says.
 
Compressing a photo into a JPEG, for example, a camera may exclude some information from the EXIF data. Similarly, when optimizing the image for the web, the data may be completely lost.
 
“Many of the photo editing tools leave traces behind," Connor notes. "An example is if you use the clone tool [in Photoshop] and clone from one area to another, then you will have a repetition of pixels.”

By carefully going over bits of the image, careful observers can spot suspect areas full of duplicate pixel groups, which are a clue to the cloning.  
 
Connor asks: “Is that field of flowers actually one small square of flowers that was cloned again and again?"

 
He says another sign of Photoshopping are halos within the image, noting that “if you apply image enhancements you may get halos around certain parts of the images.”
 
Connor’s final suggestion is to look into the shadows and reflections. “There have been studies that show that the human visual system is not particularly attuned to spotting problems with shadows. If they (the things in the images) are even remotely in the right place — or even in the wrong place — as long as there is a shadow then we tend to not see anything wrong.”
 
In his work, Dr. Farid focuses on image forensic techniques that use “geometric analysis” to find manipulations. In a series of blog posts for the Fourandsix web site, he talks about checking that shadows and reflections are consistent with light sources and that even the reflections in subject’s eyes or other parts of the image match the things around them.  
 
And as far as reflections go, that is exactly what tripped up the Moscow patriarchy. Whoever Photoshopped out the Brequet watch, simply overlooked the reflection of the watch that was right in front of their eyes.