I'm trying to emulate the stream drawing from Sources.mapJournal through IMap which receives data from IoT device. The processing of this stream is too slow and I'm getting the big accumulated outcome after 30-60 seconds.
When I started to update the IMap frequently with small data (12 KB per value), the exception is:
com.hazelcast.ringbuffer.StaleSequenceException: sequence:123 is too small and data store is disabled.
I increased the default capacity of IMap journal 10 times. It became stable after that, but very slow. A similar issue is when I'm updating the IMap with big values (about 1.2 MB per 5 seconds). Additionally I have several connected IoT devices and each of them has its own Jet job with the same pipeline:
StreamStage<TagPosition> sourceSteam =
p.drawFrom(Sources.<TagPosition, String, TagPosition>mapJournal(
Constants.IMAP_TAGS_POSITIONS_BUFFER,
Util.mapPutEvents().and(entry -> ((String) entry.getKey()).startsWith(instanceNumber)),
Util.mapEventNewValue(),
JournalInitialPosition.START_FROM_OLDEST));
// Drain to SmartMap
sourceSteam.drainTo(SmartMapSinks.newTagPositionSink(instanceNumber));
Thanks in advance!
UPD:
- The journal size is EventJournalConfig.DEFAULT_CAPACITY * 10 = 100 000 (1 partition)
- Jet version is 0.7.2
- Serialazable classes
implements
com.hazelcast.nio.serialization.IdentifiedDataSerializable