0

bq有没有办法使用标准 SQL 使用命令行工具来漂亮地格式化 Big Query“值表”结果集?这可以在标准 SQL 的 BigQuery 控制台中按预期工作,但在 bq 中则不行。我找不到方便的解决方法。它也可以使用 Legacy SQL 按预期工作。

我的用例:我正在探索 BigQuery 中包含大量列的表。BigQuery 中的匿名查询要求所选列具有唯一名称,否则您会收到消息“不支持结果中的重复列名称。发现重复:...”。当我只是在探索数据时,这是一个非常严格的要求。幸运的是,BigQuery Web 控制台中有一种方法可以通过在选择表达式中使用值表来解决这个问题......

with
left_side as
(
select 1 as id, 'hello' as col1 union all 
select 2 as id, 'there' as col1
)
,right_side as
(
select 1 as id, 'hello' as col1 union all 
select 2 as id, 'world' as col1
)
select a, b
  from left_side a
       join
       right_side b
          on a.id = b.id

...这会产生以下表格结果集,其中列自动枚举,非常好...

表格结果集

但是,在命令行上传递完全相同的查询bq如下(为方便起见,通过此处的文档)。通常,我将查询放在一个文件中并将其重定向到bq,例如,bq query --use_legacy_sql=false < bq_test4.sql

bq query --use_legacy_sql=false <<BQ_TEST
with
left_side as
(
select 1 as id, 'hello' as col1 union all 
select 2 as id, 'there' as col1
)
,right_side as
(
select 1 as id, 'hello' as col1 union all 
select 2 as id, 'world' as col1
)
select a, b
  from left_side a
       join
       right_side b
          on a.id = b.id
BQ_TEST

--format...无论传递给bq...的任何选项如何,都会产生相同的基本结果集

+---------------------------+---------------------------+
|             a             |             b             |
+---------------------------+---------------------------+
| {"id":"1","col1":"hello"} | {"id":"1","col1":"hello"} |
| {"id":"2","col1":"there"} | {"id":"2","col1":"world"} |
+---------------------------+---------------------------+

这样的结果集对我没有帮助。

好的,我可以将 json 转换为表格结果集...

with
left_side as
(
select 1 as id, 'hello' as col1 union all 
select 2 as id, 'there' as col1
)
,right_side as
(
select 1 as id, 'hello' as col1 union all 
select 2 as id, 'world' as col1
)
select json_extract_scalar(to_json_string(a), "$['id']") as `a_id`
      ,json_extract_scalar(to_json_string(a), "$['col1']") as `a_col1`
      ,json_extract_scalar(to_json_string(b), "$['id']") as `b_id`
      ,json_extract_scalar(to_json_string(b), "$['col1']") as `b_col1`
  from left_side a
       join
       right_side b
          on a.id = b.id

...这会导致,而这正是 Legacy SQL 会产生的...

+------+--------+------+--------+
| a_id | a_col1 | b_id | b_col1 |
+------+--------+------+--------+
| 1    | hello  | 1    | hello  |
| 2    | there  | 2    | world  |
+------+--------+------+--------+

必须枚举列违背了我的用例的意图。

在使用标准 SQL 时,有什么方法可以避免在加入之前或加入之后枚举列?

4

1 回答 1

1

仍然需要一些打字 - 但不那么冗长的版本

#standardSQL
WITH left_side AS (
  SELECT 1 AS id, 'hello' AS col1 UNION ALL 
  SELECT 2 AS id, 'there' AS col1
), right_side AS (
  SELECT 1 AS id, 'hello' AS col1 UNION ALL 
  SELECT 2 AS id, 'world' AS col1
)
SELECT *
FROM (SELECT NULL a_id, NULL a_col1 UNION ALL SELECT * FROM left_side) a
JOIN (SELECT NULL b_id, NULL b_col1 UNION ALL SELECT * FROM right_side) b
ON a_id = b_id

结果

Row a_id    a_col1  b_id    b_col1   
1   1       hello   1       hello    
2   2       there   2       world    

所以将在bq中如下...

+------+--------+------+--------+
| a_id | a_col1 | b_id | b_col1 |
+------+--------+------+--------+
| 1    | hello  | 1    | hello  |
| 2    | there  | 2    | world  |
+------+--------+------+--------+
于 2020-02-12T14:34:46.373 回答