The system collects many time series data which is unaligned:
Example:
+----------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-----+
| Time | 12:00 | 12:01 | 12:02 | 12:03 | 12:04 | 12:05 | 12:06 | 12:07 | 12:08 | ... |
+----------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-----+
| Series 1 | 8 | | 2 | | 4 | | 8 | | 6 | |
| Series 2 | | 5 | | 4 | | 7 | | 2 | | |
| Series 3 | 5 | | | | 7 | | | | 2 | |
| ... | | | | | | | | | | |
+----------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-----+
and may be also delayed or send in batch (i.e. the event time may not be equal to receive time, depends on the source)
The raw data will be up-sampling to 1 minute interval and fill missing value by linear interpolation with previous value. Then do the element-wised transformation like this:
Series 2 = Series 2 + Series 3
Series 1 = Series 1 * Series 2
so Series 1 depends on itself and Series 2. Series 2 depends on itself and Series 3.
The relationship of transformation between series can form a Directed acyclic graph (DAG). The relationship will change in runtime if user request for the change.
Currently all calculations are done instantly with Python Pandas when user retrieve the time series data. But the performance is getting worse when the volume of data increase or user selecting a wide time range.
Is there any way/tool to achieve this, such as stream or batch processing?
