4

我正在使用 LZ4 压缩和解压缩字符串。我尝试了以下方式

public class CompressionDemo {

    public static byte[] compressLZ4(LZ4Factory factory, String data) throws IOException {
        final int decompressedLength = data.getBytes().length;
        LZ4Compressor compressor = factory.fastCompressor();
        int maxCompressedLength = compressor.maxCompressedLength(decompressedLength);
        byte[] compressed = new byte[maxCompressedLength];
        compressor.compress(data.getBytes(), 0, decompressedLength, compressed, 0, maxCompressedLength);
        return compressed;

    }

    public static String deCompressLZ4(LZ4Factory factory, byte[] data) throws IOException {
        LZ4FastDecompressor decompressor = factory.fastDecompressor();
        byte[] restored = new byte[data.length];
        decompressor.decompress(data,0,restored, 0,data.length);
        return new String(restored);
    }

    public static void main(String[] args) throws IOException, DataFormatException {
        String string = "kjshfhshfashfhsakjfhksjafhkjsafhkjashfkjhfjkfhhjdshfhhjdfhdsjkfhdshfdskjfhksjdfhskjdhfkjsdhfk";
        LZ4Factory factory = LZ4Factory.fastestInstance();
        byte[] arr = compressLZ4(factory, string);
        System.out.println(arr.length);
        System.out.println(deCompressLZ4(factory, arr) + "decom");
    }
}

它给出了以下异常

线程“主”net.jpountz.lz4.LZ4Exception 中的异常：解码输入缓冲区的偏移量 92 时出错

这里的问题是，只有当我传递实际的字符串字节 [] 长度时，解压缩才有效，即

public static String deCompressLZ4(LZ4Factory factory, byte[] data) throws IOException {
        LZ4FastDecompressor decompressor = factory.fastDecompressor();
        byte[] restored = new byte[data.length];
        decompressor.decompress(data,0,restored, 0,"kjshfhshfashfhsakjfhksjafhkjsafhkjashfkjhfjkfhhjdshfhhjdfhdsjkfhdshfdskjfhksjdfhskjdhfkjsdhfk".getBytes().length);
        return new String(restored);
    }

它期望实际的字符串字节 [] 大小。有人可以帮我弄这个吗

4

4 回答 4

3

由于压缩和解压缩可能发生在不同的机器上，或者机器默认的字符编码不是 Unicode 格式之一，所以也应该指出编码。

对于其余部分，它使用实际的压缩和解压缩长度，并以纯格式更好地存储未压缩数据的大小，因此可以在解压缩之前对其进行提取。

public static byte[] compressLZ4(LZ4Factory factory, String data) throws IOException {
    byte[] decompressed = data.getBytes(StandardCharsets.UTF_8).length;
    LZ4Compressor compressor = factory.fastCompressor();
    int maxCompressedLength = compressor.maxCompressedLength(decompressed.length);
    byte[] compressed = new byte[4 + maxCompressedLength];
    int compressedSize = compressor.compress(decompressed, 0, decompressed.length,
                                             compressed, 4, maxCompressedLength);
    ByteBuffer.wrap(compressed).putInt(decompressed.length);
    return Arrays.copyOf(compressed, 0, 4 + compressedSize);
}

public static String deCompressLZ4(LZ4Factory factory, byte[] data) throws IOException {
    LZ4FastDecompressor decompressor = factory.fastDecompressor();
    int decrompressedLength = ByteBuffer.wrap(data).getInt();
    byte[] restored = new byte[decrompressedLength];
    decompressor.decompress(data, 4, restored, 0, decrompressedLength);
    return new String(restored, StandardCharsets.UTF_8);
}

应该说，这String不适合二进制数据，并且您的压缩/解压缩仅用于文本处理。（字符串包含 UTF-16 两字节字符形式的 Unicode 文本。转换为二进制数据总是涉及到二进制数据编码的转换。这会增加内存、速度和可能的数据损坏。）

于 2018-01-30T15:49:25.123 回答
1

我刚刚在 Android 上遇到了同样的错误，并根据以下问题解决了它： https ://github.com/lz4/lz4-java/issues/68

简而言之，确保您对两个操作（压缩+解压缩）使用相同的工厂并使用 Arrays.copyOf() 如下：

  byte[] compress(final byte[] data) {
     LZ4Factory lz4Factory = LZ4Factory.safeInstance();
     LZ4Compressor fastCompressor = lz4Factory.fastCompressor();
     int maxCompressedLength = fastCompressor.maxCompressedLength(data.length);
     byte[] comp = new byte[maxCompressedLength];
     int compressedLength = fastCompressor.compress(data, 0, data.length, comp, 0, maxCompressedLength);
     return Arrays.copyOf(comp, compressedLength);
}

  byte[] decompress(final byte[] compressed) {
     LZ4Factory lz4Factory = LZ4Factory.safeInstance();
     LZ4SafeDecompressor decompressor = lz4Factory.safeDecompressor();
     byte[] decomp = new byte[compressed.length * 4];//you might need to allocate more
     decomp = decompressor.decompress(Arrays.copyOf(compressed, compressed.length), decomp.length);
     return decomp;

希望这会有所帮助。

于 2018-01-30T15:03:37.497 回答
0

我是这样解决的：

public static byte[] decompress( byte[] finalCompressedArray,String ... extInfo) {
    int len = finalCompressedArray.length * 3;
    int i = 5;
    while (i > 0) {
        try {
            return decompress(finalCompressedArray, len);
        } catch (Exception e) {
            len = len * 2;
            i--;
            if (LOGGER.isInfoEnabled()) {
                LOGGER.info("decompress Error: extInfo ={} ", extInfo, e);
            }

        }

    }

    throw new ItemException(1, "decompress error");
}

/**
 * 解压一个数组
 *
 * @param finalCompressedArray 压缩后的数据
 * @param length               原始数据长度, 精确的长度,不能大,也不能小。
 * @return
 */
private static byte[] decompress(byte[] finalCompressedArray, int length) {
    byte[] desc = new byte[length ];
    int decompressLen = decompressor.decompress(finalCompressedArray, desc);

    byte[] result = new byte[decompressLen];
    System.arraycopy(desc,0,result,0,decompressLen);
    return result;
}
于 2016-06-08T11:41:48.753 回答
0

恢复的 byte[] 长度太小，不应该使用压缩后的 data.length，而应该使用 data[].length * 3 或大于 3。

于 2016-06-03T07:09:38.450 回答