amazon-web-services - 使用 AWS DMS 将数据从 RDS Postgres（只读副本）复制到 Redshift 时出现复制槽错误

Question

我正在尝试使用 Amazon 的数据迁移服务将数据从 RDS Postgres 的只读副本复制到 Amazon redshift。我能够成功地在同一源和目标上执行迁移任务，但在执行复制时我的 CloudWatch 上出现以下错误：您能帮我理解为什么会出现此错误吗？

2017-07-09T14:45:24 [TARGET_LOAD ]I: Going to start Redshift File Transfer client (redshift_imp.c:925)
2017-07-09T14:45:24 [SOURCE_CAPTURE ]I: Going to connect to ODBC connection string: Driver={PostgreSQL Unicode(x64)};Server=postgres-replica.1mginfra.com;Port=-199231648;Database=my_sample_db;UseDeclareFetch=1;Fetch=10000;Uid=la_la_username;Pwd=****; (postgres_endpoint_util.c:258)
2017-07-09T14:45:24 [SOURCE_UNLOAD ]I: Going to connect to ODBC connection string: Driver={PostgreSQL Unicode(x64)};Server=postgres-replica.1mginfra.com;Port=-148900000;Database=my_sample_db;UseDeclareFetch=1;Fetch=10000;Uid=la_la_username;Pwd=****; (postgres_endpoint_util.c:258)
2017-07-09T14:45:24 [TARGET_LOAD ]I: Going to connect to server blahblah.redshift.amazonaws.com database test_dms (redshift_imp.c:736)
2017-07-09T14:45:24 [TARGET_LOAD ]I: Connected to server blahblah.redshift.amazonaws.com database test_dms successfully. (redshift_imp.c:755)
2017-07-09T14:45:24 [TARGET_LOAD ]I: Creating Redshift File Transfer client (redshift_imp.c:867)
2017-07-09T14:45:24 [TARGET_LOAD ]I: Going to start Redshift File Transfer client (redshift_imp.c:925)
2017-07-09T14:45:25 [SOURCE_CAPTURE ]I: Going to connect to ODBC connection string: Driver={PostgreSQL Unicode(x64)};Server=postgres-replica.1mginfra.com;Port=-146798752;Database=my_sample_db;UseDeclareFetch=1;Fetch=10000;Uid=la_la_username;Pwd=****; (postgres_endpoint_util.c:258)
2017-07-09T14:45:25 [SOURCE_UNLOAD ]I: Going to connect to ODBC connection string: Driver={PostgreSQL Unicode(x64)};Server=postgres-replica.1mginfra.com;Port=-144697504;Database=my_sample_db;UseDeclareFetch=1;Fetch=10000;Uid=la_la_username;Pwd=****; (postgres_endpoint_util.c:258)
2017-07-09T14:45:25 [TARGET_LOAD ]I: Going to connect to server blahblah.redshift.amazonaws.com database test_dms (redshift_imp.c:736)
2017-07-09T14:45:25 [TARGET_LOAD ]I: Connected to server blahblah.redshift.amazonaws.com database test_dms successfully. (redshift_imp.c:755)
2017-07-09T14:45:25 [TARGET_LOAD ]I: Creating Redshift File Transfer client (redshift_imp.c:867)
2017-07-09T14:45:25 [TARGET_LOAD ]I: Going to start Redshift File Transfer client (redshift_imp.c:925)
2017-07-09T14:45:25 [SOURCE_CAPTURE ]I: Going to connect to ODBC connection string: Driver={PostgreSQL Unicode(x64)};Server=postgres-replica.1mginfra.com;Port=-94374048;Database=my_sample_db;UseDeclareFetch=1;Fetch=10000;Uid=la_la_username;Pwd=****; (postgres_endpoint_util.c:258)
2017-07-09T14:45:25 [SOURCE_UNLOAD ]I: Going to connect to ODBC connection string: Driver={PostgreSQL Unicode(x64)};Server=postgres-replica.1mginfra.com;Port=-44042400;Database=my_sample_db;UseDeclareFetch=1;Fetch=10000;Uid=la_la_username;Pwd=****; (postgres_endpoint_util.c:258)
2017-07-09T14:45:25 [TARGET_LOAD ]I: Going to connect to server blahblah.redshift.amazonaws.com database test_dms (redshift_imp.c:736)
2017-07-09T14:45:25 [TARGET_LOAD ]I: Connected to server blahblah.redshift.amazonaws.com database test_dms successfully. (redshift_imp.c:755)
2017-07-09T14:45:25 [TARGET_LOAD ]I: Creating Redshift File Transfer client (redshift_imp.c:867)
2017-07-09T14:45:25 [TARGET_LOAD ]I: Going to start Redshift File Transfer client (redshift_imp.c:925)
2017-07-09T14:45:25 [SOURCE_CAPTURE ]I: Going to connect to ODBC connection string: Driver={PostgreSQL Unicode(x64)};Server=postgres-replica.1mginfra.com;Port=-41941152;Database=my_sample_db;UseDeclareFetch=1;Fetch=10000;Uid=la_la_username;Pwd=****; (postgres_endpoint_util.c:258)
2017-07-09T14:45:25 [SOURCE_UNLOAD ]I: Going to connect to ODBC connection string: Driver={PostgreSQL Unicode(x64)};Server=postgres-replica.1mginfra.com;Port=-39839904;Database=my_sample_db;UseDeclareFetch=1;Fetch=10000;Uid=la_la_username;Pwd=****; (postgres_endpoint_util.c:258)
2017-07-09T14:45:25 [TARGET_LOAD ]I: Going to connect to server blahblah.redshift.amazonaws.com database test_dms (redshift_imp.c:736)
2017-07-09T14:45:25 [TARGET_LOAD ]I: Connected to server blahblah.redshift.amazonaws.com database test_dms successfully. (redshift_imp.c:755)
2017-07-09T14:45:25 [TARGET_LOAD ]I: Creating Redshift File Transfer client (redshift_imp.c:867)
2017-07-09T14:45:25 [TARGET_LOAD ]I: Going to start Redshift File Transfer client (redshift_imp.c:925)
2017-07-09T14:45:25 [SOURCE_CAPTURE ]I: Going to connect to ODBC connection string: Driver={PostgreSQL Unicode(x64)};Server=postgres-replica.1mginfra.com;Port=6289248;Database=my_sample_db;UseDeclareFetch=1;Fetch=10000;Uid=la_la_username;Pwd=****; (postgres_endpoint_util.c:258)
2017-07-09T14:45:25 [TARGET_LOAD ]I: Going to connect to server blahblah.redshift.amazonaws.com database test_dms (redshift_imp.c:736)
2017-07-09T14:45:25 [TARGET_LOAD ]I: Connected to server blahblah.redshift.amazonaws.com database test_dms successfully. (redshift_imp.c:755)
2017-07-09T14:45:25 [TARGET_LOAD ]I: Creating Redshift File Transfer client (redshift_imp.c:867)
2017-07-09T14:45:25 [TARGET_LOAD ]I: Going to start Redshift File Transfer client (redshift_imp.c:925)
2017-07-09T14:45:25 [SOURCE_CAPTURE ]I: Going to connect to ODBC connection string: Driver={PostgreSQL Unicode(x64)};Server=postgres-replica.1mginfra.com;Port=226494304;Database=my_sample_db;UseDeclareFetch=1;Fetch=10000;Uid=la_la_username;Pwd=****; (postgres_endpoint_util.c:258)
2017-07-09T14:45:25 [SOURCE_UNLOAD ]I: Going to connect to ODBC connection string: Driver={PostgreSQL Unicode(x64)};Server=postgres-replica.1mginfra.com;Port=224393056;Database=my_sample_db;UseDeclareFetch=1;Fetch=10000;Uid=la_la_username;Pwd=****; (postgres_endpoint_util.c:258)
2017-07-09T14:45:29 [TASK_MANAGER ]I: Subtask #0 ended (replicationtask_util.c:925)
2017-07-09T14:45:29 [TASK_MANAGER ]I: Subtask #1 ended (replicationtask_util.c:925)
2017-07-09T14:45:29 [TASK_MANAGER ]I: Subtask #2 ended (replicationtask_util.c:925)
2017-07-09T14:45:29 [TASK_MANAGER ]I: Subtask #3 ended (replicationtask_util.c:925)
2017-07-09T14:45:29 [TASK_MANAGER ]I: Subtask #4 ended (replicationtask_util.c:925)
2017-07-09T14:45:29 [TASK_MANAGER ]I: Subtask #5 ended (replicationtask_util.c:925)
2017-07-09T14:45:29 [TASK_MANAGER ]I: Subtask #6 ended (replicationtask_util.c:925)
2017-07-09T14:45:29 [TASK_MANAGER ]I: Task management thread terminated (replicationtask.c:2679)
2017-07-09T14:48:15 [TASK_MANAGER ]I: Task '6UJ63JGAHT7XDURNKAJPQATIXU' running CDC only in resume mode (replicationtask.c:805)
2017-07-09T14:48:15 [TASK_MANAGER ]I: Creating threads for all components (replicationtask.c:1279)
2017-07-09T14:48:15 [TASK_MANAGER ]I: CDC Control buffer stream size for QAYMAYIYZXMVMSBSGQRBYUUXQI is set to 5MB (subtask.c:772)
2017-07-09T14:48:15 [TASK_MANAGER ]I: CDC Control buffer stream size for 4UC3KDSEXMGBDPTTFU2DYBTMNE is set to 5MB (subtask.c:772)
2017-07-09T14:48:15 [TASK_MANAGER ]I: CDC Control buffer stream size for transaction_sorter is set to 5MB (subtask.c:772)
2017-07-09T14:48:15 [TASK_MANAGER ]I: Threads for all components were created (replicationtask.c:1425)
2017-07-09T14:48:15 [TASK_MANAGER ]I: Task initialization completed successfully (replicationtask.c:2265)
2017-07-09T14:48:15 [SOURCE_CAPTURE ]I: Going to connect to ODBC connection string: Driver={PostgreSQL Unicode(x64)};Server=postgres-replica.1mginfra.com;Port=-1751124128;Database=my_sample_db;UseDeclareFetch=1;Fetch=10000;Uid=la_la_username;Pwd=****; (postgres_endpoint_util.c:258)
2017-07-09T14:48:15 [TARGET_APPLY ]I: Going to connect to server blahblah.redshift.amazonaws.com database test_dms (redshift_imp.c:736)
2017-07-09T14:48:15 [TARGET_APPLY ]I: Connected to server blahblah.redshift.amazonaws.com database test_dms successfully. (redshift_imp.c:755)
2017-07-09T14:48:15 [TARGET_APPLY ]I: Creating Redshift File Transfer client (redshift_imp.c:867)
2017-07-09T14:48:16 [TARGET_APPLY ]I: Going to start Redshift File Transfer client (redshift_imp.c:925)
2017-07-09T14:48:16 [TARGET_APPLY ]I: Restore bulk state. Last bulk last record id - '0', last applied record id - '0', target confirmed record id - '0' (endpointshell.c:890)
2017-07-09T14:48:16 [TARGET_APPLY ]I: Target endpoint does not support direct insert in batch apply mode (bulk_apply.c:468)
2017-07-09T14:48:16 [TARGET_APPLY ]I: Working in bulk apply mode (endpointshell.c:897)
2017-07-09T14:48:16 [SOURCE_CAPTURE ]I: Going to connect to ODBC connection string: Driver={PostgreSQL Unicode(x64)};Server=postgres-replica.1mginfra.com;Port=-1749022880;Database=my_sample_db;UseDeclareFetch=1;Fetch=10000;Uid=la_la_username;Pwd=****; (postgres_endpoint_util.c:258)
2017-07-09T14:48:16 [TASK_MANAGER ]I: All stream components were initialized (replicationtask.c:2110)
2017-07-09T14:48:16 [SOURCE_CAPTURE ]I: Initial position is set to 'now' (postgres_endpoint_util.c:349)
2017-07-09T14:48:16 [TASK_MANAGER ]I: Starting subtask #1 (replicationtask_util.c:862)
2017-07-09T14:48:16 [SOURCE_CAPTURE ]I: Replication slot '6uj63jgaht7xdurn_00016400_ca44edb7_78eb_46e7_a992_7ffa63eb3b05' does NOT exist. (postgres_endpoint_capture.c:283)
2017-07-09T14:48:16 [SOURCE_CAPTURE ]I: Initial positioning requested is 'now' (postgres_endpoint_capture.c:285)
2017-07-09T14:48:16 [SOURCE_UNLOAD ]I: Going to connect to ODBC connection string: Driver={PostgreSQL Unicode(x64)};Server=postgres-replica.1mginfra.com;Port=-1695901856;Database=my_sample_db;UseDeclareFetch=1;Fetch=10000;Uid=la_la_username;Pwd=****; (postgres_endpoint_util.c:258)
2017-07-09T14:48:16 [SOURCE_CAPTURE ]E: Failed to execute statement: 'SELECT xlog_position FROM pg_create_logical_replication_slot('6uj63jgaht7xdurn_00016400_ca44edb7_78eb_46e7_a992_7ffa63eb3b05', 'test_decoding')' [122502] ODBC general error. (ar_odbc_stmt.c:2184)
2017-07-09T14:48:16 [TASK_MANAGER ]I: Starting subtask #2 (replicationtask_util.c:862)
2017-07-09T14:48:16 [SOURCE_CAPTURE ]E: RetCode: SQL_ERROR SqlState: 55000 NativeError: 1 Message: ERROR: logical decoding requires wal_level >= logical;, Error while executing the query [122502] ODBC general error. (ar_odbc_stmt.c:2190)
2017-07-09T14:48:16 [TASK_MANAGER ]I: Starting subtask #3 (replicationtask_util.c:862)
2017-07-09T14:48:16 [SOURCE_CAPTURE ]E: Creating replication slot '6uj63jgaht7xdurn_00016400_ca44edb7_78eb_46e7_a992_7ffa63eb3b05' failed. [122502] ODBC general error. (postgres_endpoint_capture.c:306)
2017-07-09T14:48:16 [SOURCE_CAPTURE ]E: Error executing source loop [122502] ODBC general error. (streamcomponent.c:1473)
2017-07-09T14:48:16 [TARGET_LOAD ]I: Going to connect to server blahblah.redshift.amazonaws.com database test_dms (redshift_imp.c:736)
2017-07-09T14:48:16 [TASK_MANAGER ]E: Stream component failed at subtask 0, component st_0_QAYMAYIYZXMVMSBSGQRBYUUXQI [122502] ODBC general error. (subtask.c:1380)
2017-07-09T14:48:16 [TASK_MANAGER ]I: Starting subtask #4 (replicationtask_util.c:862)
2017-07-09T14:48:16 [SOURCE_CAPTURE ]E: Stream component 'st_0_QAYMAYIYZXMVMSBSGQRBYUUXQI' terminated [122502] ODBC general error. (subtask.c:1545)
2017-07-09T14:48:16 [TASK_MANAGER ]I: Starting subtask #5 (replicationtask_util.c:862)
2017-07-09T14:48:16 [TARGET_LOAD ]I: Connected to server blahblah.redshift.amazonaws.com database test_dms successfully. (redshift_imp.c:755)
2017-07-09T14:48:16 [TARGET_LOAD ]I: Creating Redshift File Transfer client (redshift_imp.c:867)
2017-07-09T14:48:16 [TASK_MANAGER ]I: Starting subtask #6 (replicationtask_util.c:862)
2017-07-09T14:48:16 [TASK_MANAGER ]E: Task error notification received from subtask 0, thread 0 [122502] ODBC general error. (replicationtask.c:1938)
2017-07-09T14:48:16 [TASK_MANAGER ]I: Task '6UJ63JGAHT7XDURNKAJPQATIXU' encountered a recoverable error (repository.c:4107)
2017-07-09T14:48:16 [TARGET_LOAD ]I: Going to start Redshift File Transfer client (redshift_imp.c:925)
2017-07-09T14:48:16 [SOURCE_CAPTURE ]I: Going to connect to ODBC connection string: Driver={PostgreSQL Unicode(x64)};Server=postgres-replica.1mginfra.com;Port=-1465911456;Database=my_sample_db;UseDeclareFetch=1;Fetch=10000;Uid=la_la_username;Pwd=****; (postgres_endpoint_util.c:258)
2017-07-09T14:48:16 [SOURCE_UNLOAD ]I: Going to connect to ODBC connection string: Driver={PostgreSQL Unicode(x64)};Server=postgres-replica.1mginfra.com;Port=-1415579808;Database=my_sample_db;UseDeclareFetch=1;Fetch=10000;Uid=la_la_username;Pwd=****; (postgres_endpoint_util.c:258)
2017-07-09T14:48:16 [TARGET_LOAD ]I: Going to connect to server blahblah.redshift.amazonaws.com database test_dms (redshift_imp.c:736)
2017-07-09T14:48:16 [TARGET_LOAD ]I: Connected to server blahblah.redshift.amazonaws.com database test_dms successfully. (redshift_imp.c:755)
2017-07-09T14:48:16 [TARGET_LOAD ]I: Creating Redshift File Transfer client (redshift_imp.c:867)
2017-07-09T14:48:16 [TARGET_LOAD ]I: Going to start Redshift File Transfer client (redshift_imp.c:925)
2017-07-09T14:48:16 [SOURCE_CAPTURE ]I: Going to connect to ODBC connection string: Driver={PostgreSQL Unicode(x64)};Server=postgres-replica.1mginfra.com;Port=-1413478560;Database=my_sample_db;UseDeclareFetch=1;Fetch=10000;Uid=la_la_username;Pwd=****; (postgres_endpoint_util.c:258)
2017-07-09T14:48:16 [SOURCE_UNLOAD ]I: Going to connect to ODBC connection string: Driver={PostgreSQL Unicode(x64)};Server=postgres-replica.1mginfra.com;Port=-1411377312;Database=my_sample_db;UseDeclareFetch=1;Fetch=10000;Uid=la_la_username;Pwd=****; (postgres_endpoint_util.c:258)
2017-07-09T14:48:16 [TARGET_LOAD ]I: Going to connect to server blahblah.redshift.amazonaws.com database test_dms (redshift_imp.c:736)
2017-07-09T14:48:16 [TARGET_LOAD ]I: Connected to server blahblah.redshift.amazonaws.com database test_dms successfully. (redshift_imp.c:755)
2017-07-09T14:48:16 [TARGET_LOAD ]I: Creating Redshift File Transfer client (redshift_imp.c:867)
2017-07-09T14:48:16 [TARGET_LOAD ]I: Going to start Redshift File Transfer client (redshift_imp.c:925)
2017-07-09T14:48:16 [SOURCE_CAPTURE ]I: Going to connect to ODBC connection string: Driver={PostgreSQL Unicode(x64)};Server=postgres-replica.1mginfra.com;Port=-1361053856;Database=my_sample_db;UseDeclareFetch=1;Fetch=10000;Uid=la_la_username;Pwd=****; (postgres_endpoint_util.c:258)
2017-07-09T14:48:16 [SOURCE_UNLOAD ]I: Going to connect to ODBC connection string: Driver={PostgreSQL Unicode(x64)};Server=postgres-replica.1mginfra.com;Port=-1310722208;Database=my_sample_db;UseDeclareFetch=1;Fetch=10000;Uid=la_la_username;Pwd=****; (postgres_endpoint_util.c:258)
2017-07-09T14:48:16 [TARGET_LOAD ]I: Going to connect to server blahblah.redshift.amazonaws.com database test_dms (redshift_imp.c:736)
2017-07-09T14:48:16 [TARGET_LOAD ]I: Connected to server blahblah.redshift.amazonaws.com database test_dms successfully. (redshift_imp.c:755)
2017-07-09T14:48:16 [TARGET_LOAD ]I: Creating Redshift File Transfer client (redshift_imp.c:867)
2017-07-09T14:48:16 [TARGET_LOAD ]I: Going to start Redshift File Transfer client (redshift_imp.c:925)
2017-07-09T14:48:16 [SOURCE_CAPTURE ]I: Going to connect to ODBC connection string: Driver={PostgreSQL Unicode(x64)};Server=postgres-replica.1mginfra.com;Port=-1308620960;Database=my_sample_db;UseDeclareFetch=1;Fetch=10000;Uid=la_la_username;Pwd=****; (postgres_endpoint_util.c:258)
2017-07-09T14:48:16 [SOURCE_UNLOAD ]I: Going to connect to ODBC connection string: Driver={PostgreSQL Unicode(x64)};Server=postgres-replica.1mginfra.com;Port=-1306519712;Database=my_sample_db;UseDeclareFetch=1;Fetch=10000;Uid=la_la_username;Pwd=****; (postgres_endpoint_util.c:258)
2017-07-09T14:48:16 [TARGET_LOAD ]I: Going to connect to server blahblah.redshift.amazonaws.com database test_dms (redshift_imp.c:736)
2017-07-09T14:48:16 [TARGET_LOAD ]I: Connected to server blahblah.redshift.amazonaws.com database test_dms successfully. (redshift_imp.c:755)
2017-07-09T14:48:16 [TARGET_LOAD ]I: Creating Redshift File Transfer client (redshift_imp.c:867)
2017-07-09T14:48:17 [TARGET_LOAD ]I: Going to start Redshift File Transfer client (redshift_imp.c:925)
2017-07-09T14:48:17 [SOURCE_CAPTURE ]I: Going to connect to ODBC connection string: Driver={PostgreSQL Unicode(x64)};Server=postgres-replica.1mginfra.com;Port=-1260390560;Database=my_sample_db;UseDeclareFetch=1;Fetch=10000;Uid=la_la_username;Pwd=****; (postgres_endpoint_util.c:258)
2017-07-09T14:48:17 [SOURCE_UNLOAD ]I: Going to connect to ODBC connection string: Driver={PostgreSQL Unicode(x64)};Server=postgres-replica.1mginfra.com;Port=-1042286752;Database=my_sample_db;UseDeclareFetch=1;Fetch=10000;Uid=la_la_username;Pwd=****; (postgres_endpoint_util.c:258)
2017-07-09T14:48:17 [TARGET_LOAD ]I: Going to connect to server blahblah.redshift.amazonaws.com database test_dms (redshift_imp.c:736)
2017-07-09T14:48:17 [TARGET_LOAD ]I: Connected to server blahblah.redshift.amazonaws.com database test_dms successfully. (redshift_imp.c:755)
2017-07-09T14:48:17 [TARGET_LOAD ]I: Creating Redshift File Transfer client (redshift_imp.c:867)
2017-07-09T14:48:17 [TARGET_LOAD ]I: Going to start Redshift File Transfer client (redshift_imp.c:925)
2017-07-09T14:48:17 [SOURCE_CAPTURE ]I: Going to connect to ODBC connection string: Driver={PostgreSQL Unicode(x64)};Server=postgres-replica.1mginfra.com;Port=-1040185504;Database=my_sample_db;UseDeclareFetch=1;Fetch=10000;Uid=la_la_username;Pwd=****; (postgres_endpoint_util.c:258)
2017-07-09T14:48:20 [TASK_MANAGER ]I: Subtask #0 ended (replicationtask_util.c:925)
2017-07-09T14:48:20 [TASK_MANAGER ]I: Subtask #1 ended (replicationtask_util.c:925)
2017-07-09T14:48:20 [TASK_MANAGER ]I: Subtask #2 ended (replicationtask_util.c:925)
2017-07-09T14:48:20 [TASK_MANAGER ]I: Subtask #3 ended (replicationtask_util.c:925)
2017-07-09T14:48:20 [TASK_MANAGER ]I: Subtask #4 ended (replicationtask_util.c:925)
2017-07-09T14:48:20 [TASK_MANAGER ]I: Subtask #5 ended (replicationtask_util.c:925)
2017-07-09T14:48:20 [TASK_MANAGER ]I: Subtask #6 ended (replicationtask_util.c:925)
2017-07-09T14:48:20 [TASK_MANAGER ]I: Task management thread terminated (replicationtask.c:2679)
2017-07-09T14:53:45 [TASK_MANAGER ]I: Task '6UJ63JGAHT7XDURNKAJPQATIXU' running CDC only in resume mode (replicationtask.c:805)
2017-07-09T14:53:45 [TASK_MANAGER ]I: Creating threads for all components (replicationtask.c:1279)
2017-07-09T14:53:45 [TASK_MANAGER ]I: CDC Control buffer stream size for QAYMAYIYZXMVMSBSGQRBYUUXQI is set to 5MB (subtask.c:772)
2017-07-09T14:53:45 [TASK_MANAGER ]I: CDC Control buffer stream size for 4UC3KDSEXMGBDPTTFU2DYBTMNE is set to 5MB (subtask.c:772)
2017-07-09T14:53:45 [TASK_MANAGER ]I: CDC Control buffer stream size for transaction_sorter is set to 5MB (subtask.c:772)
2017-07-09T14:53:45 [TASK_MANAGER ]I: Threads for all components were created (replicationtask.c:1425)
2017-07-09T14:53:45 [TASK_MANAGER ]I: Task initialization completed successfully (replicationtask.c:2265)
2017-07-09T14:53:45 [SOURCE_CAPTURE ]I: Going to connect to ODBC connection string: Driver={PostgreSQL Unicode(x64)};Server=postgres-replica.1mginfra.com;Port=-727713952;Database=my_sample_db;UseDeclareFetch=1;Fetch=10000;Uid=la_la_username;Pwd=****; (postgres_endpoint_util.c:258)
2017-07-09T14:53:45 [TARGET_APPLY ]I: Going to connect to server blahblah.redshift.amazonaws.com database test_dms (redshift_imp.c:736)
2017-07-09T14:53:45 [TARGET_APPLY ]I: Connected to server blahblah.redshift.amazonaws.com database test_dms successfully. (redshift_imp.c:755)
2017-07-09T14:53:45 [TARGET_APPLY ]I: Creating Redshift File Transfer client (redshift_imp.c:867)
2017-07-09T14:53:45 [TARGET_APPLY ]I: Going to start Redshift File Transfer client (redshift_imp.c:925)
2017-07-09T14:53:45 [TARGET_APPLY ]I: Restore bulk state. Last bulk last record id - '0', last applied record id - '0', target confirmed record id - '0' (endpointshell.c:890)
2017-07-09T14:53:45 [TARGET_APPLY ]I: Target endpoint does not support direct insert in batch apply mode (bulk_apply.c:468)
2017-07-09T14:53:45 [TARGET_APPLY ]I: Working in bulk apply mode (endpointshell.c:897)
2017-07-09T14:53:45 [SOURCE_CAPTURE ]I: Going to connect to ODBC connection string: Driver={PostgreSQL Unicode(x64)};Server=postgres-replica.1mginfra.com;Port=-725612704;Database=my_sample_db;UseDeclareFetch=1;Fetch=10000;Uid=la_la_username;Pwd=****; (postgres_endpoint_util.c:258)
2017-07-09T14:53:45 [TASK_MANAGER ]I: All stream components were initialized (replicationtask.c:2110)
2017-07-09T14:53:45 [SOURCE_CAPTURE ]I: Initial position is set to 'now' (postgres_endpoint_util.c:349)
2017-07-09T14:53:45 [TASK_MANAGER ]I: Starting subtask #1 (replicationtask_util.c:862)
2017-07-09T14:53:45 [SOURCE_UNLOAD ]I: Going to connect to ODBC connection string: Driver={PostgreSQL Unicode(x64)};Server=postgres-replica.1mginfra.com;Port=-672491680;Database=my_sample_db;UseDeclareFetch=1;Fetch=10000;Uid=la_la_username;Pwd=****; (postgres_endpoint_util.c:258)
2017-07-09T14:53:45 [SOURCE_CAPTURE ]I: Replication slot '6uj63jgaht7xdurn_00016400_ca44edb7_78eb_46e7_a992_7ffa63eb3b05' does NOT exist. (postgres_endpoint_capture.c:283)
2017-07-09T14:53:45 [SOURCE_CAPTURE ]I: Initial positioning requested is 'now' (postgres_endpoint_capture.c:285)
2017-07-09T14:53:45 [SOURCE_CAPTURE ]E: Failed to execute statement: 'SELECT xlog_position FROM pg_create_logical_replication_slot('6uj63jgaht7xdurn_00016400_ca44edb7_78eb_46e7_a992_7ffa63eb3b05', 'test_decoding')' [122502] ODBC general error. (ar_odbc_stmt.c:2184)
2017-07-09T14:53:45 [SOURCE_CAPTURE ]E: RetCode: SQL_ERROR SqlState: 55000 NativeError: 1 Message: ERROR: logical decoding requires wal_level >= logical;, Error while executing the query [122502] ODBC general error. (ar_odbc_stmt.c:2190)
2017-07-09T14:53:45 [SOURCE_CAPTURE ]E: Creating replication slot '6uj63jgaht7xdurn_00016400_ca44edb7_78eb_46e7_a992_7ffa63eb3b05' failed. [122502] ODBC general error. (postgres_endpoint_capture.c:306)
2017-07-09T14:53:46 [TASK_MANAGER ]I: Starting subtask #2 (replicationtask_util.c:862)
2017-07-09T14:53:46 [TASK_MANAGER ]I: Starting subtask #3 (replicationtask_util.c:862)
2017-07-09T14:53:46 [TASK_MANAGER ]I: Starting subtask #4 (replicationtask_util.c:862)
2017-07-09T14:53:46 [TARGET_LOAD ]I: Going to connect to server blahblah.redshift.amazonaws.com database test_dms (redshift_imp.c:736)
2017-07-09T14:53:46 [SOURCE_CAPTURE ]E: Error executing source loop [122502] ODBC general error. (streamcomponent.c:1473)
2017-07-09T14:53:46 [TASK_MANAGER ]E: Stream component failed at subtask 0, component st_0_QAYMAYIYZXMVMSBSGQRBYUUXQI [122502] ODBC general error. (subtask.c:1380)
2017-07-09T14:53:46 [TASK_MANAGER ]I: Starting subtask #5 (replicationtask_util.c:862)
2017-07-09T14:53:46 [TARGET_LOAD ]I: Connected to server blahblah.redshift.amazonaws.com database test_dms successfully. (redshift_imp.c:755)
2017-07-09T14:53:46 [TARGET_LOAD ]I: Creating Redshift File Transfer client (redshift_imp.c:867)

score 2 · Accepted Answer

为了更新 wal_level 并消除错误，您需要更改 RDS 配置。更新它的最简单方法是通过控制台（网站）

登录并在 RDS 服务页面上，单击“参数组”
如果您没有更改任何内容，您可能仍在使用默认组。就我而言，我使用的是 default.aurora-postgresql12。默认组不允许更改，因此您需要创建一个新的参数组
为组命名、描述并从您的默认组复制设置
编辑组并过滤“rds.logical_replication”
修改该项目并将其值设置为 1（从 0 开始），然后保存更改
接下来更新您的集群以使用您的新参数组。这将询问您何时要应用更改。现在应用它们“应该”对集群没有影响，但你总是可以等待应用它们。无论何时应用更改，您都需要重新启动读取器/写入器实例（这花了我大约 60 秒）。
重新启动后，尝试重新启动 DMS 任务，它应该可以工作。

score 0 · Accepted Answer

此外，9.5.4 及更高版本似乎支持复制。 https://forums.aws.amazon.com/message.jspa?messageID=744635

score 0 · Accepted Answer

不支持 PG 上的只读副本上的 AWS DMS。为了启用 CDC，您需要触发 wal 创建，并通过至少执行一次备份来完成此操作，但只读副本不支持备份，因此它是一种死锁。

我已经在 AWS 中打开了一个案例

amazon-web-services - 使用 AWS DMS 将数据从 RDS Postgres（只读副本）复制到 Redshift 时出现复制槽错误

3 回答 3

Related

Reference