java - Java serialization of very large objects

Question

I'm facing the following issue: I would like to serialize a very large object (several hundreds of MB in memory) to a file. As I understood that Kryo is one of the best serialization libraries out there, I've been using this to serialize my object:

    OutputStream fOutStream = new FileOutputStream(ParamsProvider.STORAGE_HH_FILENAME);
    Output out = new Output(fOutStream);
    kryo.writeObject(out, data);
    out.close();
    fOutStream.close();

This generates an OutOfMemory exception: I guess that the entire object is first serialized in memory before being written to file.

Hence my question: is there a way / library to serialize an entire object while it is written to file per chunks?

I would like to avoid the workaround of decomposing the object in smaller elements before serializing as:

• I would like the serialization implementation to be independent of the data object structure
• My object contains multiple references to the same objects. I suspect that if I serialize those elements independently, the serializer will instantiate them as different objects (consuming even more memory)

UPDATE: In the meanwhile I implemented a serialization approach where the different elements of the very large object are serialized one by one and this does indeed avoid the OutOfMemory exception.

However I'm afraid that with this approach I will loose advantage (in terms of memory footprint) of multiple references to a same object (i.e. those references will have their own instance of the object). Any idea on this?

New code snippet (where, for instance, field1 contains references to same objects than field2):

            OutputStream fOutStream = new FileOutputStream(ParamsProvider.STORAGE_HH_FILENAME);
            Output out = new Output(fOutStream);

            kryo.writeObject(out, data.field1);
            kryo.writeObject(out, data.field2);
(...)
            kryo.writeObject(out, data.field50);
            out.close();
            fOutStream.close();

Any hint / help would be greatly appreciated! Thanks, Tom