1

我一直在成功使用 ColdFusion 函数 CSVtoArray 以编程方式将 CSV 文件转换为 Excel 文件,然后我可以通过 CFSpreadsheet 将这些文件导入我的数据库。最近我遇到了一个数据问题,该函数阻塞了,而且我对 RegEx 不够强大(我怀疑我需要)来修复它。我希望这里有人可以给我一个适当的修复。

CSV 数据通常以逗号分隔并由引号分隔,如下所示(省略号表示行前后的附加数据),其转换不会出错:

... "明尼阿波利斯","MN","55555","患者转诊 - John Smith", ...

我遇到的问题是逗号分隔的行中有多余的引号。例如,此行将失败并停止进一步导入:

... ,"明尼阿波利斯","MN","55555","患者转诊 - Robert " Bob " Smith", ...

认为我需要做的是在引号字符上运行某种正则表达式替换——但只有在它之前或之后没有逗号的地方?但我不知道如何做这样的事情。有什么建议么?

我将在下面包含我正在使用的完整 CSVtoArray 函数代码。我将不胜感激任何提示/修复。谢谢!

    <cffunction
name="csvToArray"
access="public"
returntype="array"
output="false"
hint="I take a CSV file or CSV data value and convert it to an array of arrays based on the given field delimiter. Line delimiter is assumed to be new line / carriage return related.">
 
<!--- Define arguments. --->
<cfargument
name="file"
type="string"
required="false"
default=""
hint="I am the optional file containing the CSV data."
/>
 
<cfargument
name="csv"
type="string"
required="false"
default=""
hint="I am the CSV text data (if the file argument was not used)."
/>
 
<cfargument
name="delimiter"
type="string"
required="false"
default=","
hint="I am the field delimiter (line delimiter is assumed to be new line / carriage return)."
/>
 
<cfargument
name="trim"
type="boolean"
required="false"
default="true"
hint="I flags whether or not to trim the END of the file for line breaks and carriage returns."
/>
 
<!--- Define the local scope. --->
<cfset var local = {} />
 
<!---
Check to see if we are using a CSV File. If so, then all we
want to do is move the file data into the CSV variable. That
way, the rest of the algorithm can be uniform.
--->
<cfif len( arguments.file )>

<cfset FilePathName="\\ply-fileserver\EPDE\DOWNLOADS\#arguments.file#">

<!--- Read the file into Data. --->
<cfset arguments.csv = fileRead( FilePathName ) />
 
</cfif>
 
<!---
ASSERT: At this point, no matter how the data was passed in,
we now have it in the CSV variable.
--->
 
<!---
Check to see if we need to trim the data. Be default, we are
going to pull off any new line and carraige returns that are
at the end of the file (we do NOT want to strip spaces or
tabs as those are field delimiters).
--->
<cfif arguments.trim>
 
<!--- Remove trailing line breaks and carriage returns. --->
<cfset arguments.csv = reReplace(
arguments.csv,
"[\r\n]+$",
"",
"all"
) />
 
</cfif>
 
<!--- Make sure the delimiter is just one character. --->
<cfif (len( arguments.delimiter ) neq 1)>
 
<!--- Set the default delimiter value. --->
<cfset arguments.delimiter = "," />
 
</cfif>
 
 
<!---
Now, let's define the pattern for parsing the CSV data. We
are going to use verbose regular expression since this is a
rather complicated pattern.
 
NOTE: We are using the verbose flag such that we can use
white space in our regex for readability.
--->
<cfsavecontent variable="local.regEx">(?x)
<cfoutput>
 
<!--- Make sure we pick up where we left off. --->
\G
 
<!---
We are going to start off with a field value since
the first thing in our file should be a field (or a
completely empty file).
--->
(?:
 
<!--- Quoted value - GROUP 1 --->
"([^"]*+ (?>""[^"]*+)* )"
 
|
 
<!--- Standard field value - GROUP 2 --->
([^"\#arguments.delimiter#\r\n]*+)
 
)
 
<!--- Delimiter - GROUP 3 --->
(
\#arguments.delimiter# |
\r\n? |
\n |
$
)
 
</cfoutput>
</cfsavecontent>
 
<!---
Create a compiled Java regular expression pattern object
for the experssion that will be parsing the CSV.
--->
<cfset local.pattern = createObject(
"java",
"java.util.regex.Pattern"
).compile(
javaCast( "string", local.regEx )
)
/>
 
<!---
Now, get the pattern matcher for our target text (the CSV
data). This will allows us to iterate over all the tokens
in the CSV data for individual evaluation.
--->
<cfset local.matcher = local.pattern.matcher(
javaCast( "string", arguments.csv )
) />
 
 
<!---
Create an array to hold the CSV data. We are going to create
an array of arrays in which each nested array represents a
row in the CSV data file. We are going to start off the CSV
data with a single row.
 
NOTE: It is impossible to differentiate an empty dataset from
a dataset that has one empty row. As such, we will always
have at least one row in our result.
--->
<cfset local.csvData = [ [] ] />
 
<!---
Here's where the magic is taking place; we are going to use
the Java pattern matcher to iterate over each of the CSV data
fields using the regular expression we defined above.
 
Each match will have at least the field value and possibly an
optional trailing delimiter.
--->
<cfloop condition="local.matcher.find()">
 
<!---
Next, try to get the qualified field value. If the field
was not qualified, this value will be null.
--->
<cfset local.fieldValue = local.matcher.group(
javaCast( "int", 1 )
) />
 
<!---
Check to see if the value exists in the local scope. If
it doesn't exist, then we want the non-qualified field.
If it does exist, then we want to replace any escaped,
embedded quotes.
--->
<cfif structKeyExists( local, "fieldValue" )>
 
<!---
The qualified field was found. Replace escpaed
quotes (two double quotes in a row) with an unescaped
double quote.
--->
<cfset local.fieldValue = replace(
local.fieldValue,
"""""",
"""",
"all"
) />
 
<cfelse>
 
<!---
No qualified field value was found; as such, let's
use the non-qualified field value.
--->
<cfset local.fieldValue = local.matcher.group(
javaCast( "int", 2 )
) />
 
</cfif>
 
<!---
Now that we have our parsed field value, let's add it to
the most recently created CSV row collection.
--->
<cfset arrayAppend(
local.csvData[ arrayLen( local.csvData ) ],
local.fieldValue
) />
 
<!---
Get the delimiter. We know that the delimiter will always
be matched, but in the case that it matched the end of
the CSV string, it will not have a length.
--->
<cfset local.delimiter = local.matcher.group(
javaCast( "int", 3 )
) />
 
<!---
Check to see if we found a delimiter that is not the
field delimiter (end-of-file delimiter will not have
a length). If this is the case, then our delimiter is the
line delimiter. Add a new data array to the CSV
data collection.
--->
<cfif (
len( local.delimiter ) &&
(local.delimiter neq arguments.delimiter)
)>
 
<!--- Start new row data array. --->
<cfset arrayAppend(
local.csvData,
arrayNew( 1 )
) />
 
<!--- Check to see if there is no delimiter length. --->
<cfelseif !len( local.delimiter )>
 
<!---
If our delimiter has no length, it means that we
reached the end of the CSV data. Let's explicitly
break out of the loop otherwise we'll get an extra
empty space.
--->
<cfbreak />
 
</cfif>
 
</cfloop>
 
 
<!---
At this point, our array should contain the parsed contents
of the CSV value as an array of arrays. Return the array.
--->
<cfreturn local.csvData />
</cffunction>
4

1 回答 1

-1

提示编号 1

I've been using the ColdFusion function CSVtoArray successfully to programmatically convert CSV files into Excel files that I can then import into my database via CFSpreadsheet. 这对我来说听起来像是过度工程。您是否考虑过读取文件,<cffile>然后遍历结果变量以插入记录?

提示编号 2

您不需要正则表达式来处理引号。您可以使用更简单的字符串函数,如下所示:

// delete leading double quote
row = replace(row, '"', '', 'one');
// delete trailing double quote
row = left(row, len(row) -1);
// change csv delimiters to ascii 50, record separator
row = replace(row, '",", chr(50), 'all');

在插入查询中使用查询参数将处理任何其他双引号或单引号。

于 2020-07-09T17:07:35.817 回答