python - How can I set up an automated import to Google Data Prep?

Question

When using Google Data Prep, I am able to create automated schedules to run jobs that update my BigQuery tables.

However, this seems pointless when considering that the data used in Prep is updated by manually dragging and dropping CSVs (or JSON, xlsx, whatever) into the data storage bucket.

I have attempted to search for a definitive way of updating this bucket automatically with files that are regularly updated on my PC, but there seems to be no best-practice solution that I can find.

How should one go about doing this efficiently and effectively?

Answer 1

因此，为了将文件从您的计算机上传到 Google Cloud Storage，有几种可能性。如果您只是运行一个处理该共享目录中任何更改的守护进程，您可以使用以下不同语言编写自动上传代码：C#、Go、Java、Node.JS、PHP、Python 或 Ruby。

您在此处有一些用于上传对象的代码示例，但请注意，还有详细的云存储客户端库参考，您还可以在“其他资源”中找到 GitHub 链接。