FourMatch works by comparing the “signature” for an image against a database of known signatures from a large variety of cameras, mobile devices, software, and on-line services. What exactly is an image signature, and why is it an effective indicator of the authenticity of an image?
Nearly all cameras save images in the JPEG format. Though this is a universal format, it affords a nearly endless variety to exactly how a JPEG file is constructed and stored. This variety results in a highly distinctive set of signatures from each hardware and software product. Because modern-day devices store the camera’s make and model in an image’s metadata, we can compare the signature of an image against a database of known camera-specific signatures. When a signature correctly matches a known signature from the device that captured the photo, you can be confident that the photo has not been edited.
What’s in a signature?
We’ve collected some of the most distinctive aspects of a camera’s JPEG format into our concept of a signature. We’ve grouped these aspects into four main components of a FourMatch signature:
- The first, and simplest, component is simply the image dimensions. The dimensions distinguish between devices with varying sensor size and resolution settings (regardless of image orientation—landscape vs. portrait). If you are presented with an image that was recorded from a particular camera, but the size of the image is different than what you know the camera can capture, then it’s a safe bet that the image is not an original.
- JPEG is known as a lossy compression scheme which means that it reduces the quality of the recorded image in return for a reduced file size. There are hundreds of parameters that specify precisely how this compression is achieved, and each camera and software manufacturer customizes these parameters to suit their needs. Due to the large variety of possible parameters, the JPEG compression settings are the most distinctive aspect of the image signature.
- A three-channel color image (RGB) is first converted into a three-channel luminance/chrominance image (YCbCr).
- Each image channel is partitioned into non-overlapping 8x8 pixel blocks.
- A discrete cosine transform (DCT) is applied to each 8x8 block of each image channel. The DCT effectively reorganizes the pixel block into regions of low, medium, and high frequency content. This partitioning allows for less perceptually salient image content (high frequencies) to be eliminated or reduced in quality while retaining the quality of the more perceptually salient information (low frequencies).
- The DCT values from the previous step are quantized in value (divided by an integer and then converted from a floating point value into an integer). The larger the divisor the more significant the compression and loss of information. By allowing each DCT value within an 8x8 block to be quantized by a different amount, the compression can be selectively applied to more or less salient image content. The quantization within an image channel is the same for each 8x8 block, but is typically different for each image channel. The JPEG standard does not specify by how much DCT values should be quantized. Instead, it is left to the engineers to decide, allowing them to find a compromise between image quality and the amount of compression. This customization leads to an enormous variety in exactly how each device compresses an image.
- The quantized DCT values are then subjected to run-length encoding. This final loss-less compression step encodes the more frequently occurring quantized values with the fewest number of bits, and the less frequently occurring values with more bits. Although the JPEG standard does employ a standard algorithm for this step (Huffman coding), there remains some degree of variability in precisely how the encoding is chosen, which allows for further customization and distinctiveness.
- Many devices create and store a thumbnail sized image at the time that the full resolution image is recorded. This thumbnail is used for quick previewing of the image. The thumbnail is typically stored as a second JPEG with its own compression values within the main JPEG file. Thus, the third component of the signature is the thumbnail size and compression parameters. The lack of a thumbnail, although less distinct than the actual thumbnail parameters, is still considered to be a distinctive feature of a device.
- Modern-day devices store a variety of image and device data in the image’s EXIF metadata. The EXIF standard specifies five main directories into which this data is stored (data can also be stored in additional directories specified by the manufacturer). The standard does not, however, specify precisely what information should be stored here. As such, each camera manufacturer is free to store as much or as little information as desired. The fourth component of the signature is the number of data fields stored in each of these five directories. This part of the signature eschews the actual data fields and data for a more compact and equatable representation of the metadata.
Click to learn more.
How does JPEG compression work?
The JPEG compression of an image consists of five main steps:
The color-converted, DCT-transformed, quantized, and Huffman encoded values are stored in the resulting JPEG file. In order to decompress a file, the encoding parameters (image size, quantization values, and Huffman codes) are also stored. It is from these stored parameters, and the image metadata, that the identifying signature is constructed.
As we’ve described above, the JPEG signature consists of four basic components that each characterize various aspects of how a JPEG image was created and saved. We have built a database of signatures from millions of images spanning thousands of cameras, mobile devices, software, and on-line services. These signatures, while not perfectly unique, are highly distinct. Once an image has been edited and re-saved from a software product, this signature is changed to match the software rather than the original capture device. By comparing an image’s signature against our database, we can instantly determine if it is an original.