1

我正在尝试使用 curl 将包含大型 base64 编码文件（大小：400Mb）的 json 上传到 Spring REST 接口。但我收到 OutOfMemory 错误。我知道包含该文件的 json 超出了 JVM 的堆大小。什么方法可以处理这么大的文件？任何示例处理代码将不胜感激。

外壳命令：

jsonstring='{"uuid":"111","type":"REPORT","userdata":"test defined 
data","time":3000,"wasted":120,"status":"PASS","message":"demo 
message","report":"'"$(base64 file.zip)"'"}'
curl -s --insecure -H "Content-Type: application/json;" --data "@-" 
https://localhost:443/api/v1/upload <<<"$jsonstring"

弹簧休息接口：

@RequestMapping(value = "/api/v1/upload", method = RequestMethod.POST)
public String uploadFile(@RequestBody final UploadedFile uploadedFile, final 
HttpServletRequest request,
final HttpServletResponse response) {
byte[] decoded = 
DatatypeConverter.parseBase64Binary(uploadedFile.getReport());

错误 ：

Jul 17 10:03:38  bash[29824]: 2018-07-17 10:03:38.686 ERROR 29829 --- [.0-31443-exec-8] o.a.c.c.C.[.[.[/].[dispatcherServlet]    : Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Handler dispatch failed; nested exception is java.lang.OutOfMemoryError: Java heap space] with root cause
Jul 17 10:03:38 chld9004852 bash[29824]: java.lang.OutOfMemoryError: Java heap space
Jul 17 10:03:38 chld9004852 bash[29824]: at java.lang.AbstractStringBuilder.<init>(AbstractStringBuilder.java:68) ~[na:1.8.0_152]
Jul 17 10:03:38 chld9004852 bash[29824]: at java.lang.StringBuilder.<init>(StringBuilder.java:101) ~[na:1.8.0_152]
Jul 17 10:03:38 chld9004852 bash[29824]: at com.fasterxml.jackson.core.util.TextBuffer.contentsAsString(TextBuffer.java:394) ~[jackson-core-2.9.6.jar!/:2.9.6]
Jul 17 10:03:38 chld9004852 bash[29824]: at com.fasterxml.jackson.core.json.UTF8StreamJsonParser._finishAndReturnString(UTF8StreamJsonParser.java:2408) ~[jackson-core-2.9.6.jar!/:2.9.6]
Jul 17 10:03:38 chld9004852 bash[29824]: at com.fasterxml.jackson.core.json.UTF8StreamJsonParser.getText(UTF8StreamJsonParser.java:269) ~[jackson-core-2.9.6.jar!/:2.9.6]
Jul 17 10:03:38 chld9004852 bash[29824]: at com.fasterxml.jackson.databind.deser.std.StringDeserializer.deserialize(StringDeserializer.java:35) ~[jackson-databind-2.9.6.jar!/:2.9.6]
Jul 17 10:03:38 chld9004852 bash[29824]: at com.fasterxml.jackson.databind.deser.std.StringDeserializer.deserialize(StringDeserializer.java:10) ~[jackson-databind-2.9.6.jar!/:2.9.6]
Jul 17 10:03:38 chld9004852 bash[29824]: at com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeAndSet(MethodProperty.java:127) ~[jackson-databind-2.9.6.jar!/:2.9.6]
Jul 17 10:03:38 chld9004852 bash[29824]: at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:369) ~[jackson-databind-2.9.6.jar!/:2.9.6]
4

1 回答 1

2

您的主要问题是 Spring 使用 Jackson 映射器将整个内容加载到内存中。您只能通过使用较低级别的Streaming API JSON 解析器来避免这种情况——想想 XML SAX 或 StAX 与 DOM。

这个想法是让您的控制器期望一个InputStream而不是映射对象并手动解析它，一次一个令牌。这样，您就可以只创建处理该请求所需的对象或数据结构，而不是加载整个事物以及相当多的开销。

当然，另一种选择是增加 JVM 的可用堆，并等待下一个大文件使其崩溃。;)

于 2018-07-18T06:29:20.947 回答