Firstly - Thibaut, thank you for Kiba. It goes toe-to-toe with 'enterprise' grade ETL tools and has never let me down.
I'm busy building an ETL pipeline that takes a numbers of rows, and reduces them down into a single summary row. I get the feeling that this should be a simple thing, but I'm a little stumped on how to approach this problem.
We have a number of CDR's from a voice switch, and need to condense them under some simple criteria into a a handful of summary records. So, the problem is; I have many thousands of records coming in from a Source
, and need to transform them into only a few records based on some reduce criteria.
Kiba is really simple when there's a one-to-one Source
-> Destination
ETL, or even a one-to-many Source
-> Destination
with the new enumerable exploder in V3, but I don't see a clear path to many-to-one ETL pipelines.
Any suggestions or guidance would be greatly appreciated.