0

我尝试在 Windows 中使用时间戳模式创建源 Kafka 连接。我有这张桌子:

CREATE TABLE "usu"."mytable" (  
"first_name" CHAR(8 BYTE) DEFAULT ' ',  
"last_name" CHAR(8 BYTE) DEFAULT ' ',  
"regist" TIMESTAMP (0) DEFAULT SYS_EXTRACT_UTC(SYSTIMESTAMP)  
) TABLESPACE "temp" ;  

oracle数据库属性:

name=jdbc-conector  
connector.class=io.confluent.connect.jdbc.JdbcSourceConnector  
connection.url=jdbc:oracle:thin:@localhost:1521:xe  
connection.user=usu  
connection.password=pwd  
query=select NDZ, to_timestamp(STRING_TIMESTAMP,'YYYY - MM - DD HH: MI: SS, FF6') AS TIMESTAMP_COLUMN FROM myTable
mode=timestamp+incrementing 
timestamp.column.name=TIMESTAMP_COLUMN
incrementing.column.name=NDZ
db.timezone=UTC
dialect.name=OracleDatabaseDialect
numeric.mapping=best_fit
table.types=TABLE  
poll.interval.ms=1000  
topic.prefix=input-mytable

INSERT INTO "usu"."mytable" (first_name,last_name) values("jake","tyler");
select output:
jake    tyler   2019-08-23 11:54:47:046

控制台消费者不返回任何输出:

kafka-console-consumer --bootstrap-server localhost:9092 --topic input-mytable  

连接器仅在批量模式下工作。我不懂为什么。

检查连接器:

curl localhost:8083/connectors/jdbc-conector/status/  
{"name":"jdbc-conector","connector": 
{"state":"RUNNING","worker_id":"1.2.3.4:8083"},"tasks": 
[{"id":0,"state":"RUNNING","worker_id":"1.2.3.4:8083"}],"type":"source"}

Ed:在调试模式下,我的输出是这样的。

源卡夫卡连接:

D:\kafka\bin\windows\connect-standalone.bat D:\kafka\config\connect-standalone.properties D:\kafka\config\connect-bbdd.properties  

输出:

DEBUG: prepared SQL query: 'select NDZ, to_timestamp(STRING_TIMESTAMP,'YYYY - MM - DD HH: MI: SS, FF6') AS TIMESTAMP_COLUMN FROM myTable) WHERE "TIMESTAMP_COLUMN" < ? AND(("TIMESTAMP_COLUMN" = ? AND "NDZ" > ? ) OR "TIMESTAMP_COLUMN" > ? ) ORDER BY "TIMESTAMP_COLUMN", "NDZ" ASC.  

[2019-08-23 14:01:36,150] DEBUG Executing prepared statement with start time value = 2019-08-23 11:40:47:046 end time = 2019-08-23 11:40:47:046 and incrementing value = 19 (io.confluent.connect.jdbc.source.TimestampIncrementingCriteria)  

如何控制时差?

4

1 回答 1

0

对我来说,这个问题与我的 kafka 连接器的时区有关。我的 Kafka 连接器是 GMT 时间,数据库是 GMT+3。所以所有插入\更新都提前了,因此连接器无法看到它们。

于 2020-10-12T13:40:08.710 回答