我有一个用 java 编写的 Azure 函数,它将在 azure 上侦听队列消息,队列消息具有 azure blob 容器上 zip 文件的路径,一旦收到队列消息,它会从 azure 上的路径位置获取 zip 文件并解压缩到天蓝色的容器。它适用于小尺寸文件,但 > 80 MB 它显示FailureException: OutOfMemoryError: Java heap spaceStack
异常。我的代码如下
@FunctionName("queueprocessor")
public void run(@QueueTrigger(name = "msg",
queueName = "queuetest",
dataType = "",
connection = "AzureWebJobsStorage") Details message,
final ExecutionContext executionContext,
@BlobInput(name = "file",
dataType = "binary",
connection = "AzureWebJobsStorage",
path = "{Path}") byte[] content) {
executionContext.getLogger().info("PATH: " + message.getPath());
CloudStorageAccount storageAccount = null;
CloudBlobClient blobClient = null;
CloudBlobContainer container = null;
try {
String connectStr = "DefaultEndpointsProtocol=https;AccountName=name;AccountKey=mykey;EndpointSuffix=core.windows.net";
//unique name of the container
String containerName = "output";
// Config to upload file size > 1MB in chunks
int deltaBackoff = 2;
int maxAttempts = 2;
BlobRequestOptions blobReqOption = new BlobRequestOptions();
blobReqOption.setSingleBlobPutThresholdInBytes(1024 * 1024); // 1MB
blobReqOption.setRetryPolicyFactory(new RetryExponentialRetry(deltaBackoff, maxAttempts));
// Parse the connection string and create a blob client to interact with Blob storage
storageAccount = CloudStorageAccount.parse(connectStr);
blobClient = storageAccount.createCloudBlobClient();
blobClient.setDefaultRequestOptions(blobReqOption);
container = blobClient.getContainerReference(containerName);
container.createIfNotExists(BlobContainerPublicAccessType.CONTAINER, new BlobRequestOptions(), new OperationContext());
ZipInputStream zipIn = new ZipInputStream(new ByteArrayInputStream(content));
ZipEntry zipEntry = zipIn.getNextEntry();
while (zipEntry != null) {
executionContext.getLogger().info("ZipEntry name: " + zipEntry.getName());
//Getting a blob reference
CloudBlockBlob blob = container.getBlockBlobReference(zipEntry.getName());
ByteArrayOutputStream outputB = new ByteArrayOutputStream();
byte[] buf = new byte[1024];
int n;
while ((n = zipIn.read(buf, 0, 1024)) != -1) {
outputB.write(buf, 0, n);
}
// Upload to container
ByteArrayInputStream inputS = new ByteArrayInputStream(outputB.toByteArray());
blob.setStreamWriteSizeInBytes(256 * 1024); // 256K
blob.upload(inputS, inputS.available());
executionContext.getLogger().info("ZipEntry name: " + zipEntry.getName() + " extracted");
zipIn.closeEntry();
zipEntry = zipIn.getNextEntry();
}
zipIn.close();
executionContext.getLogger().info("FILE EXTRACTION FINISHED");
} catch(Exception e) {
e.printStackTrace();
}
}
Details message
具有 ID 和文件路径,路径作为@BlobInput(..., path ={Path},...)
. 根据我的分析,我觉得@BlobInput
正在将完整文件加载到内存中,这就是我得到OutOfMemoryError
. 如果我是对的,请让我知道任何其他方式来避免它?因为将来文件大小可能会达到 2GB。如果解压缩代码有任何错误,请告诉我。谢谢。