8

我想用'|'替换没有包含在双引号中的每个逗号 中使用模式匹配。

例如,给定以下输入:

A,B,"C,D",E,"F,G",H,"I,J,K"
"Chang, Yao-Jen",33,MIS,"Taiwan, Taipei",M

这是所需的输出:

A|B|"C,D"|E|"F,G"|H|"I,J,K"
"Chang, Yao-Jen"|33|MIS|"Taiwan, Taipei"|M

我希望在不使用硬编码的情况下完成此操作,例如:

sed '2s/33,MIS/33|MIS|/' file.
4

4 回答 4

13

第一个样本:又快又脏:

如果您的逗号总是后跟文本字符串中的空格,并且从不在字段分隔中,您可以使用:

sed -e 's/,\([^ ]\)/\|\1/g'
"Chang, Yao-Jen"|33|MIS|"Taiwan, Taipei"|M

但你必须确定一个字符

更精致的样品,无需空间,最接近您的原始想法。

sed -e ':a;s/^\(\("[^"]*"\|[^",]*\)*\),/\1|/;ta'

echo '"Chang, Yao-Jen",33,MIS,"Taiwan, Taipei",M' |
  sed -e ':a;s/^\(\("[^"]*"\|[^",]*\)*\),/\1|/;ta'
"Chang, Yao-Jen"|33|MIS|"Taiwan, Taipei"|M

echo '"Chang,Yao-Jen",33,MIS,"Taiwan,Taipei",M' |
  sed -e '1 { :a;s/^\(\("[^"]*"\|[^",]*\)*\),/\1|/;ta }'
"Chang,Yao-Jen"|33|MIS|"Taiwan,Taipei"|M

解释:

sed -e '
    :a
    s/^\(\("[^"]*"\|[^",]*\)*\),/\1|/
    ta
'
  • :a是分支(循环)的地址位置
  • s/从行首搜索 '[^",]*,' 或 '"...",' 而不是用 vbar 替换逗号。
  • tas/如果先前已匹配,则分支到 a 。

由于您要求在第 2 行进行操作,您必须:

sed -e '2 { :a; s/^\(\("[^"]*"\|[^",]*\)*\),/\1|/; ta } '

编辑: [错了!见编辑 3 ]

如果您想要混合引号和双引号,则为另一个示例:

有一个示例包含混合引用、未引用和一个 包含引号但双引号的字段:

cat <<eof >sample
A,B,"C,D",E,"F,G",H,"I,J,K"
"Chang, Yao-Jen",33,MIS,"Taiwan, Taipei",M
A,B,'C,D',E,'F,G',H,'I,J,K'
'Chang, Yao-Jen',33,MIS,'Taiwan, Taipei',M
"Chang, Yao-Jen",33,MIS,"Taiwan, Taipei",M,'Chang,Yao-Jen',34,MZZ,'Taiwan, Taipei',Z
"Chang's son: Yao-Lu",55,MAA,'Taiwan, too',z
eof

sed -e ':a;s/^\(\(\(['\''"]\)[^\3]*\3\|[^",'\'']*\)*\),/\1|/;ta' sample
A|B|"C,D"|E|"F,G"|H|"I,J,K"
"Chang, Yao-Jen"|33|MIS|"Taiwan, Taipei"|M
A|B|'C,D'|E|'F,G'|H|'I,J,K'
'Chang, Yao-Jen'|33|MIS|'Taiwan, Taipei'|M
"Chang, Yao-Jen"|33|MIS|"Taiwan, Taipei"|M|'Chang,Yao-Jen'|34|MZZ|'Taiwan, Taipei'|Z
"Chang's son: Yao-Lu"|55|MAA|'Taiwan, too'|z

其中 sed 脚本可以限制在一个更具可读性的脚本文件中:

cat <<oesedscript >csvtopsv.sed 
#!/bin/sed -f 
# Coma Separated Values to Pipe Separated Values
:a
s/^\(\(\(['"]\)[^\3]*\3\|[^",']*\)*\),/\1|/;
ta
oesedscript
chmod +x csvtopsv.sed

./csvtopsv.sed sample
A|B|"C,D"|E|"F|G"|H|"I|J|K"
"Chang, Yao-Jen"|33|MIS|"Taiwan, Taipei"|M
A|B|'C,D'|E|'F|G'|H|'I|J|K'
'Chang, Yao-Jen'|33|MIS|'Taiwan, Taipei'|M
"Chang, Yao-Jen"|33|MIS|"Taiwan, Taipei"|M|'Chang,Yao-Jen'|34|MZZ|'Taiwan, Taipei'|Z
"Chang's son: Yao-Lu"|55|MAA|'Taiwan, too'|z

解释:

s/搜索引号或双引号作为第三个封闭的['"]正则表达式部分,后跟 0 个或更多其他字符,而不是数学第三个封闭部分,最后跟一个与第三个正则表达式部分相同的第二个字符......或者没有逗号,单引号或双引号[,'"].. .

编辑 3警告!这是错误的!

所以正确的答案似乎肯定是这样的:

sed -e ':a;s/^\(\(\(['\''"]\)[^\3]*\3\|[^",'\'']*\)*\),/\1|/;ta'

;L您可以在添加调试之前看到我的错误ta

sed -e ':a;s/^\(\(\(['\''"]\)[^\3]*\3\|[^",'\'']*\)*\),/\1|/;L;ta'

在哪里

echo '1,"John Doe","6, rue Peuh",236,"B,-,F,H,P,-",-55' |
  sed -e ':a;s/^\(\("[^"]*"\|'\''[^'\'']*'\''\|[^",'\'']*\)*\),/\1#/;L;ta'
1#"John Doe","6, rue Peuh",236,"B,-,F,H,P,-",-55
1#"John Doe"#"6, rue Peuh",236,"B,-,F,H,P,-",-55
1#"John Doe"#"6, rue Peuh"#236,"B,-,F,H,P,-",-55
1#"John Doe"#"6, rue Peuh"#236#"B,-,F,H,P,-",-55
1#"John Doe"#"6, rue Peuh"#236#"B,-,F,H,P,-"#-55
1#"John Doe"#"6, rue Peuh"#236#"B,-,F,H,P,-"#-55
1#"John Doe"#"6, rue Peuh"#236#"B,-,F,H,P,-"#-55

我们可以看到这并不那么简单......[^\3]不要给出预期的效果,而是匹配not char3

最后,我们必须自己搜索每个分隔符:

:a;
s/^\(\("[^"]*"\|'[^']*'\|[^",']*\)*\),/\1\t/;
ta

注意:从那里开始,我将以逗号形式呈现csv2tsv制表符分隔值,如果您真的更喜欢使用|管道作为分隔符,您可以替换\t|或任何您想要的字符。

命令行不那么性感:

echo '1,"John Doe","6, rue Peuh",236,"B,-,F,H,P,-",-55' |
  sed -e ':a;s/^\(\("[^"]*"\|'\''[^'\'']*'\''\|[^",'\'']*\)*\),/\1\t/;L;ta' 
1       "John Doe","6, rue Peuh",236,"B,-,F,H,P,-",-55
1       "John Doe"      "6, rue Peuh",236,"B,-,F,H,P,-",-55
1       "John Doe"      "6, rue Peuh"   236,"B,-,F,H,P,-",-55
1       "John Doe"      "6, rue Peuh"   236     "B,-,F,H,P,-",-55
1       "John Doe"      "6, rue Peuh"   236     "B,-,F,H,P,-"   -55
1       "John Doe"      "6, rue Peuh"   236     "B,-,F,H,P,-"   -55
1   "John Doe"      "6, rue Peuh"   236     "B,-,F,H,P,-"   -55

但这符合需要。

echo '1,"John Doe","6, rue Peuh",236,"B,-,F,H,P,-",-55' |
  sed -e ':a;s/^\(\("[^"]*"\|'\''[^'\'']*'\''\|[^",'\'']*\)*\),/\1\t/;ta' 
1       "John Doe"      "6, rue Peuh"   236     "B,-,F,H,P,-"   -55

很好,创建sedscript

cat >csv2tsv.sed <<eof
#!/bin/sed -f
# Coma separated values to Tab separated values

:a
s/^\(\("[^"]*"\|'[^']*'\|[^",']*\)*\),/\1\t/;
ta
eof

chmod +x csv2tsv.sed

现在:

cat >file.csv <<eof
A,B,"C,D",E,"F,G",H,"I,J,K"
"Chang, Yao-Jen",33,MIS,"Taiwan, Taipei",M
1,"John Doe","6, rue Peuh",236,"B,-,F,H,P,-",-55
4,"hacker's string",'one quote: "I have no special talents. I am only passionat\
ely curious." - Albert Einstein',unquoted string,9,1,1,3
eof

./csv2tsv.sed file.csv 
A   B       "C,D"   E      "F,G"    H    "I,J,K"
"Chang, Yao-Jen"    33     MIS      "Taiwan, Taipei"        M
1   "John Doe"      "6, rue Peuh"   236  "B,-,F,H,P,-"      -55
4   "hacker's string"      'one quote: "I have no special talents. I am only pa
ssionately curious." - Albert Einstein' unquoted string 9  1    1       3
于 2012-11-20T14:54:20.183 回答
7

这是使用GNU awkFPAT变量的一种方法:

awk 'BEGIN { FPAT="([^,]+)|(\"[^\"]+\")"; OFS="|" } $1=$1' file

结果:

A|B|"C,D"|E|"F,G"|H|"I,J,K"
"Chang, Yao-Jen"|33|MIS|"Taiwan, Taipei"|M
于 2012-11-20T15:43:51.603 回答
4
$ awk 'BEGIN{FS=OFS="\""} {for (i=1;i<=NF;i+=2) gsub(/,/,"|",$i)}1' file
A|B|"C,D"|E|"F,G"|H|"I,J,K"
"Chang, Yao-Jen"|33|MIS|"Taiwan, Taipei"|M
于 2012-11-21T16:59:30.443 回答
2

除非这是关于 sed 的学习练习,否则我会使用带有适当 CSV 解析器的语言,例如:

ruby -rcsv -ne '
    puts CSV.generate_line(CSV.parse_line($_), {:col_sep => "|"})
' filename

输出

A|B|C,D|E|F,G|H|I,J,K
Chang, Yao-Jen|33|MIS|Taiwan, Taipei|M

引号消失了。这是因为没有需要引用的“内部”分隔符。如果输入中出现了一些管道,那么您将在输出中看到一些双引号字段。

于 2012-11-20T17:01:03.660 回答