bigdata - Kylo 中的数据清理

Question

我想知道是否有任何方法可以在 Kylo ( https://kylo.io/ ) 中进行数据清理。我能够让该工具使用数据验证规则指出错误，但我很想知道它是否还可以执行其他功能。例子：

• 删除数据之间的任何空记录
• 检测和删除数据中的重复列

Answer 1

Data cleansing is handled in Kylo using standardizers and validators. However, as of Kylo 0.9.0 there's no built-in functions for removing empty rows or duplicate columns. The current functions are limited to removing rows if a specific column is empty and removing duplicate rows.

This functionality could be added by writing a plugin: http://kylo.readthedocs.io/en/latest/developer-guides/PluginApiIndex.html