java - 读取 JPG 文件的 XMP 元数据

Question

我正在开发应该利用 Google 相机的新深度图生成功能的 Android 应用程序。

基本上谷歌已经描述了这里使用的元数据

我可以访问大部分元数据，但不幸的是最重要的数据被编码为extendedXmp，我无法获得任何XMP解析库来正确解析它！

我尝试过 Commons-Imaging、元数据提取器和最近的 Adob​​es XMPCore

XMPCore 可能能够处理扩展版本，但没有文档如何让它解析 JPG 文件中的数据，它假设要传递原始 XMP 数据

是否有任何正确的 XMP 解析实现，包括 JPG 文件的扩展部分，或者我只是做错了什么？

这是我的尝试：

使用 Commons-Imaging：

                try {
                    String imageParser = new JpegImageParser().getXmpXml(new ByteSourceInputStream(imageStream, "img.jpg"), new HashMap<String, Object>());

                    Log.v(TAG, imageParser);

                } catch (ImageReadException e1) {
                    // TODO Auto-generated catch block
                    e1.printStackTrace();
                }

使用元数据提取器

                Metadata metadata = ImageMetadataReader.readMetadata(
                        new BufferedInputStream(imageStream), false);


                XmpDirectory xmp = metadata
                        .getDirectory(XmpDirectory.class);
                XMPMeta xmpMeta = xmp.getXMPMeta();



                String uri = "http://ns.google.com/photos/1.0/depthmap/";

                Log.v(TAG, xmpMeta.doesPropertyExist(uri, "GDepth:Format") + " " );

                try {
                    XMPProperty hasExtendedXMP = xmpMeta.getProperty("http://ns.adobe.com/xmp/note/", "xmpNote:HasExtendedXMP");

                    Log.v(TAG, hasExtendedXMP.getValue().toString() + " " + new String(Base64.decode(hasExtendedXMP.getValue().toString(), Base64.DEFAULT)));

                } catch (XMPException e) {
                    e.printStackTrace();
                }

Answer 1

Initially, Adobe didn't expect the XMP data length would exceed the limit of one JPEG segment (about 64K) and their XMP specification stated the XMP data must fit into one. Later when they found a single JPEG APP1 segment is not large enough to hold the XMP data, they changed their specification to allow for multiple APP1 segments for the whole XMP data. The data is split into two parts: the standard XMP and the ExtendedXMP. The standard XMP part is a "normal" XMP structure with a package wrapper while the ExtendedXMP part doesn't have a package wrapper. The ExtendedXMP data can be further divided to fit into multiple APP1.

The following quote is from Adobe XMP specification Part 3 for ExtendedXMP chunks as JPEG APP1:

Each chunk is written into the JPEG file within a separate APP1 marker segment. Each ExtendedXMP marker segment contains:

• A null-terminated signature string of "http://ns.adobe.com/xmp/extension/".
• A 128-bit GUID stored as a 32-byte ASCII hex string, capital A-F, no null termination. The GUID is a 128-bit MD5 digest of the full ExtendedXMP serialization.
• The full length of the ExtendedXMP serialization as a 32-bit unsigned integer
• The offset of this portion as a 32-bit unsigned integer.
• The portion of the ExtendedXMP

We can see besides the null-terminated string as an id for the ExtendedXMP data, there is also a GUID which should be the same value as the one found in the standard XMP part. The offset is used to join the different parts of the ExtendedXMP - so the sequence for the ExtendedXMP APP1 may not even be in order. Then come the actual data part and this is why @Matt's answer need some way to fix the string. There is another value - full length of the ExtendedXMP serialization which serves two purposes: check the integrity of the data as well as provides the buffer size for joining the data.

When we found a ExtendedXMP segment, we need to join the current data with the other ExtendedXMP segments and finally got the whole ExtendedXMP data. We then join the two XML tree together (removing the GUID from the standard XMP part as well) to retrieve the entire XMP data.

I have made a library icafe in Java which can extract and insert XMP as well as ExtendedXMP. One of the usecase for the ExtendedXMP is for Google's depth map data which in fact is a grayscale image hidden inside the actual image as a metadata, and in the case of JPEG, as XMP data. The depth map image could be used for example to blur the original image. The depth map data are usually large and have to be split into standard and extended XMP parts. The whole data is Base64 encoded and could be in PNG format.

The following is an example image and the extracted depth map:

The original image comes from here.

Note: Recently I found another website talking about Google Cardboard Camera app which can take advantage of both the image and audio embedded in the JPEG XMP data. ICAFE now supports both image and audio extraction from such images. Example usage can be found here with the following call JPEGTweaker.extractDepthMap()

Here is the image extracted by ICAFE from the original image on the website talking about Google Cardboard Camera app:

Unfortunately, I can't find a way to insert the MP4 audio here.

Answer 2

我已经能够使用元数据提取器库和通过 XMP 属性的迭代器读取也存储在 XMP 中的 Picasa 人脸数据：

try {
    Metadata metadata = ImageMetadataReader.readMetadata(imageFile);
    XmpDirectory xmpDirectory = metadata.getDirectory(XmpDirectory.class);
    XMPMeta xmpMeta = xmpDirectory.getXMPMeta();
    XMPIterator itr = xmpMeta.iterator();
    while (itr.hasNext()) {
        XMPPropertyInfo pi = (XMPPropertyInfo) itr.next();
        if (pi != null && pi.getPath() != null) {
            if ((pi.getPath().endsWith("stArea:w")) || (pi.getPath().endsWith("mwg-rs:Name")) || (pi.getPath().endsWith("stArea:h")))
                System.out.println(pi.getValue().toString());
        }
    }
} catch (final NullPointerException npe) {
  // ignore
}

Answer 3

我遇到了同样的问题，我认为问题在于扩展数据存储在第二个 xmpmeta 部分中，例如元数据提取器会跳过该部分。所以我能做的是在字节流中搜索每个部分，看看它是否具有我期望的属性。我还发现，至少对于深度图数据，base 64 编码的字符串显然被分成大约 64 KB 的部分，并且包含一些需要删除的标头才能正确解码字符串。下面的 fixString 函数很可能被知道分块信息的人替换。这依赖于https://www.adobe.com/devnet/xmp.html上提供的 xmpcore 库。

import java.io.*;
import java.util.*;
import com.adobe.xmp.*;
import com.adobe.xmp.impl.*;

public class XMP
{
    // An encoding should really be specified here, and for other uses of getBytes!
    private static final byte[] OPEN_ARR = "<x:xmpmeta".getBytes();
    private static final byte[] CLOSE_ARR = "</x:xmpmeta>".getBytes();

    private static void copy(InputStream in, OutputStream out) throws IOException
    {
        int len = -1;
        byte[] buf = new byte[1024];
        while((len = in.read(buf)) >= 0)
        {
            out.write(buf, 0, len);
        }

        in.close();
        out.close();
    }

    private static int indexOf(byte[] arr, byte[] sub, int start)
    {
        int subIdx = 0;

        for(int x = start;x < arr.length;x++)
        {
            if(arr[x] == sub[subIdx])
            {
                if(subIdx == sub.length - 1)
                {
                    return x - subIdx;
                }
                subIdx++;
            }
            else
            {
                subIdx = 0;
            }
        }

        return -1;
    }

    private static String fixString(String str)
    {
        int idx = 0;
        StringBuilder buf = new StringBuilder(str);
        while((idx = buf.indexOf("http")) >= 0)
        {
            buf.delete(idx - 4, idx + 75);
        }

        return buf.toString();
    }

    private static String findDepthData(File file) throws IOException, XMPException
    {
        FileInputStream in = new FileInputStream(file);
        ByteArrayOutputStream out = new ByteArrayOutputStream();

        copy(in, out);
        byte[] fileData = out.toByteArray();

        int openIdx = indexOf(fileData, OPEN_ARR, 0);
        while(openIdx >= 0)
        {
            int closeIdx = indexOf(fileData, CLOSE_ARR, openIdx + 1) + CLOSE_ARR.length;

            byte[] segArr = Arrays.copyOfRange(fileData, openIdx, closeIdx);
            XMPMeta meta = XMPMetaFactory.parseFromBuffer(segArr);

            String str = meta.getPropertyString("http://ns.google.com/photos/1.0/depthmap/", "Data");

            if(str != null)
            {
                return fixString(str);
            }

            openIdx = indexOf(fileData, OPEN_ARR, closeIdx + 1);
        }

        return null;
    }

    public static void main(String[] args) throws Exception
    {
        String data = findDepthData(new File(args[0]));
        if(data != null)
        {
            byte[] imgData = Base64.decode(data.getBytes());
            ByteArrayInputStream in = new ByteArrayInputStream(imgData);
            FileOutputStream out = new FileOutputStream(new File("out.png"));
            copy(in, out);
        }
    }
}