回答我自己的问题:
这里有两种可能性:输出上的 gunzip(例如 use GunzipOutputStream
,Java API 未提供),或输入上的 gunzip(例如 use GZIPInputStream
,由 Java API 提供)加上在读取期间强制执行 Content-Length。
我两者都做过,而且我认为我更喜欢后者,因为 a) 它不需要启动单独的线程来将字节从PipedOutputStream
aPipedIOnputStream
和 b) (推论,我猜)它没有这样的种族威胁-条件和其他同步问题。
首先,这是我的 实现LimitedInputStream
,它允许我包装输入流并强制限制读取的数据量。请注意,我还有一个BigLimitedInputStream
使用BigInteger
计数来支持大于的 Content-Length 值Long.MAX_LONG
:
public class LimitedInputStream
extends InputStream
{
private long _limit;
private long _read;
private InputStream _in;
public LimitedInputStream(InputStream in, long limit)
{
_limit= limit;
_in = in;
_read = 0;
}
@Override
public int available()
throws IOException
{
return _in.available(); // sure?
}
@Override
public void close()
throws IOException
{
_in.close();
}
@Override
public boolean markSupported()
{
return false;
}
@Override
public int read()
throws IOException
{
int read = _in.read();
if(-1 == read)
return -1;
++_read;
if(_read > _limit)
return -1;
// throw new IOException("Read limit reached: " + _limit);
return read;
}
@Override
public int read(byte[] b)
throws IOException
{
return read(b, 0, b.length);
}
@Override
public int read(byte[] b, int off, int len)
throws IOException
{
// 'len' is an int, so 'max' is an int; narrowing cast is safe
int max = (int)Math.min((long)(_limit - _read), (long)len);
if(0 == max && len > 0)
return -1;
//throw new IOException("Read limit reached: " + _limit);
int read = _in.read(b, off, max);
_read += read;
// This should never happen
if(_read > _limit)
return -1;
//throw new IOException("Read limit reached: " + _limit);
return read;
}
@Override
public long skip(long n)
throws IOException
{
long max = Math.min((long)(_limit - _read), n);
if(0 == max)
return 0;
long read = _in.skip(max);
_read += read;
return read;
}
}
使用上面的类来包装从InputStream
获得的HttpURLConnection
允许我简化现有代码,我必须读取Content-Length
标题中提到的精确字节数,然后盲目地将输入复制到输出。然后我将输入流(已经包装在 中LimitedInputStream
)包装在 a 中GZIPInputStream
以进行解压缩,并将字节从(双重包装)输入泵送到输出。
不太直截了当的路线是继续我原来的路线:使用(结果是)一个尴尬的类来包装 OutputStream GunzipOutputStream
:. 我编写了一个GunzipOutputStream
使用内部线程通过一对管道流泵送字节的方法。它很丑,而且它基于OpenRDF 的GunzipOutputStream
. 我觉得我的比较简单:
public class GunzipOutputStream
extends OutputStream
{
final private Thread _pump;
// Streams
final private PipedOutputStream _zipped; // Compressed bytes are written here (by clients)
final private PipedInputStream _pipe; // Compressed bytes are read (internally) here
final private OutputStream _out; // Uncompressed data is written here (by the pump thread)
// Internal state
private IOException _e;
public GunzipOutputStream(OutputStream out)
throws IOException
{
_zipped = new PipedOutputStream();
_pipe = new PipedInputStream(_zipped);
_out = out;
_pump = new Thread(new Runnable() {
public void run() {
InputStream in = null;
try
{
in = new GZIPInputStream(_pipe);
pump(in, _out);
}
catch (IOException e)
{
_e = e;
System.err.println(e);
_e.printStackTrace();
}
finally
{
try { in.close(); } catch (IOException ioe)
{ ioe.printStackTrace(); }
}
}
private void pump(InputStream in, OutputStream out)
throws IOException
{
long count = 0;
byte[] buf = new byte[4096];
int read;
while ((read = in.read(buf)) >= 0) {
System.err.println("===> Pumping " + read + " bytes");
out.write(buf, 0, read);
count += read;
}
out.flush();
System.err.println("===> Pumped a total of " + count + " bytes");
}
}, "GunzipOutputStream stream pump " + GunzipOutputStream.this.hashCode());
_pump.start();
}
public void close() throws IOException {
throwIOException();
_zipped.close();
_pipe.close();
_out.close();
}
public void flush() throws IOException {
throwIOException();
_zipped.flush();
}
public void write(int b) throws IOException {
throwIOException();
_zipped.write(b);
}
public void write(byte[] b) throws IOException {
throwIOException();
_zipped.write(b);
}
public void write(byte[] b, int off, int len) throws IOException {
throwIOException();
_zipped.write(b, off, len);
}
public String toString() {
return _zipped.toString();
}
protected void finish()
throws IOException
{
try
{
_pump.join();
_pipe.close();
_zipped.close();
}
catch (InterruptedException ie)
{
// Ignore
}
}
private void throwIOException()
throws IOException
{
if(null != _e)
{
IOException e = _e;
_e = null; // Clear the existing error
throw e;
}
}
}
同样,这可行,但它似乎相当......脆弱。
最后,我重构了我的代码以使用LimitedInputStream
andGZIPInputStream
而没有使用GunzipOutputStream
. 如果 Java API 提供了GunzipOutputStream
,那就太好了。但事实并非如此,而且如果不编写“本机”gunzip 算法,实现您自己的算法就会GunzipOutputStream
扩展适当的限制。