0

我已按照 microsoft 文档获取会话日志。https://docs.microsoft.com/en-us/azure/data-factory/copy-activity-log 这是我的复制活动的片段

“logSettings”：{“enableCopyActivityLog”：true，“copyActivityLogSettings”：{“logLevel”：“Info”，“enableReliableLogging”：false}，“logLocationSettings”：{“linkedServiceName”：{“referenceName”：“ls_adlsgen2_landingzone”，“类型”：“LinkedServiceReference”}，“路径”：“会话日志”}

作为复制活动输出的一部分，我期望日志文件具有数据读取、写入、行读取、写入、复制持续时间、吞吐量等，但是我看到的只是文件写入的开始和结束。我错过了什么吗？看起来会话日志没有记录复制文件的见解..有没有办法得到这些细节？

输出会话日志文件 Timestamp,Level,OperationName,OperationItem,Message 2021-11-18 21:30:36.3698967,Info,FileWrite,"202111181630.parquet","开始写入文件。" 2021-11-18 21:30:37.5773470,Info,FileWrite,"202111181630.parquet","完成写入文件。文件复制成功。"

4

1 回答 1

0

根据官方文档，日志文件的架构如下所示

在此处输入图像描述

不幸的是，您只能在日志文件中获取Timestamp、Level、OperationName、OperationItem、Message 列。

示例日志文件

Timestamp, Level, OperationName, OperationItem, Message
2020-10-19 08:39:13.6688152,Info,FileRead,"sample1.csv","Start to read file: {""Path"":""sample1.csv"",""ItemType"":""File"",""Size"":104857620,""LastModified"":""2020-10-19T08:22:31Z"",""ETag"":""\""0x8D874081F80C01A\"""",""ContentMD5"":""dGKVP8BVIy6AoTtKnt+aYQ=="",""ObjectName"":null}"
2020-10-19 08:39:56.3190846, Warning, FileSkip, "sample1.csv", "File is skipped after read 548000000 bytes: ErrorCode=DataConsistencySourceDataChanged,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Source file 'sample1.csv' is changed by other clients during the copy activity run.,Source=,'." 
2020-10-19 08:40:13.6688152,Info,FileRead,"sample2.csv","Start to read file: {""Path"":""sample2.csv"",""ItemType"":""File"",""Size"":104857620,""LastModified"":""2020-10-19T08:22:31Z"",""ETag"":""\""0x8D874081F80C01A\"""",""ContentMD5"":""dGKVP8BVIy6AoTtKnt+aYQ=="",""ObjectName"":null}"
2020-10-19 08:40:13.9003981,Info,FileWrite,"sample2.csv","Start to write file from source file: sample2.csv."
2020-10-19 08:45:17.6508407,Info,FileRead,"sample2.csv","Complete reading file successfully. "
2020-10-19 08:45:28.7390083,Info,FileWrite,"sample2.csv","Complete writing file from source file: sample2.csv. File is successfully copied."

更新 -

不幸的是，我找不到任何关于数据传输指标的见解

您可以在此处请求此功能

于 2021-11-20T11:58:21.680 回答