0

将（空间）数据添加到 GeoMesa/Accumulo 堆栈的最佳方法是什么？

(1) 如果我理解正确，应该创建一个 SimpleFeature 创建文件和转换器文件以添加数据。数据本身存储为 CSV。我是否正确，我们必须为我们希望添加的每个 CSV 构建这些文件？

(2) 下面的例子正确吗？例如，CSV 文件中的几何图形存储如下。“多行字符串（（2.0116069 48.9172785,2.0116474 48.9172131,2.0117161 48.917135,2.011814 48.9170714,2.0118996 48.9170489））”

（3）我们如何将这些转换器文件添加到将数据添加到GeoMesa/Accumulo堆栈的过程中？

最后的目标是有一个（简单的）过程来将数据添加到堆栈中，并在下一步中通过地理服务器打开数据。

欢迎任何形式的帮助。提前致谢。

简单的特征创建文件：

geomesa.sfts.links_geom = {
      attributes = [
        { name = "id", type = "Long" }
        { name = "length",     type = "Float" }
        { name = "number", type = "Integer" }
        ...
        { name = "geom", type = "MultiLineString", srid = 4326 }
      ]
} ```


Converter file:
geomesa.converters.links_geom = {
      type = "delimited-text",
      format = "CSV",
      id-field = "toString($id)",
      fields = [
        { name = "id", transform = "$1::long" }
        { name = "length",     transform = "$2::float" }
        { name = "number", transform = "$3::int" }
        ...
        { name = "geom", transform = "multilinestring($11)" }
      ]
}
4

1 回答 1

1

There is no "best" way to ingest data into GeoMesa, it depends on your specific use-case. The command-line tools provide an easy entry point, but more advanced scenarios might use Apache NiFi, a stream processing framework like Apache Storm, or cloud native tools like AWS Lambda.

GeoMesa is a GeoTools data store, so you can write data using the DataStore API, without any Converter definitions. There are examples of this in the geomesa-tutorials project. However, Converters provide a declarative way to define your data type without any code. They can also be re-used across environments, so if you develop a Converter for the CLI tools, you can easily use the same definition in e.g. Apache NiFi, allowing you to scale and migrate your ingest as needed.

In general, with Converters you do need to define one per file format. GeoMesa offers type inference for CSV files as described here, which may let you ingest your data without a converter, or at least provide an initial template that you can tweak to your needs. There is information on adding your Converters to the classpath here and here.

When developing an initial Converter definition, it can be helpful to use the convert CLI command, with the error mode to "raise-errors" as described here. Once your definition is solid, you can then proceed with ingestion.

于 2019-03-29T17:53:09.850 回答