是否可以通过 java jdbc 连接从 S3 到 Redshift 触发复制命令?
示例:从 's3://' 复制测试 CREDENTIALS 'aws_access_key_id=xxxxxxx;aws_secret_access_key=xxxxxxxxx'
是否可以通过 java jdbc 连接从 S3 到 Redshift 触发复制命令?
示例:从 's3://' 复制测试 CREDENTIALS 'aws_access_key_id=xxxxxxx;aws_secret_access_key=xxxxxxxxx'
是的,尝试如下代码
String dbURL = "jdbc:postgresql://x.y.us-east-1.redshift.amazonaws.com:5439/dev";
String MasterUsername = "userame";
String MasterUserPassword = "password";
Connection conn = null;
Statement stmt = null;
try{
//Dynamically load postgresql driver at runtime.
Class.forName("org.postgresql.Driver");
System.out.println("Connecting to database...");
Properties props = new Properties();
props.setProperty("user", MasterUsername);
props.setProperty("password", MasterUserPassword);
conn = DriverManager.getConnection(dbURL, props);
stmt = conn.createStatement();
String sql="copy test from 's3://' CREDENTIALS 'aws_access_key_id=xxxxxxx;aws_secret_access_key=xxxxxxxxx'"
int j = stmt.executeUpdate(sql);
stmt.close();
conn.close();
}catch(Exception ex){
//For convenience, handle all errors here.
ex.printStackTrace();
}
Sandesh 的答案非常好,但它使用 PostgreSql 驱动程序。AWS 提供 Redshift 驱动,优于 PostgreSql 驱动。其余的事情将保持不变。我希望这些信息可以帮助其他人。
1)JDBC Driver 将从org.postgresql.Driver
变为 com.amazon.redshift.jdbcXX.Driver
,其中 XX 是 Redshift 驱动的版本。例如42
。
2)Jdbc url 将从postgreSQL
变为redshift
.
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.Statement;
import java.util.Properties;
public class RedShiftJDBC {
public static void main(String[] args) {
Connection conn = null;
Statement statement = null;
try {
//Make sure to choose appropriate Redshift Jdbc driver and its jar in classpath
Class.forName("com.amazon.redshift.jdbc42.Driver");
Properties props = new Properties();
props.setProperty("user", "username***");
props.setProperty("password", "password****");
System.out.println("\n\nconnecting to database...\n\n");
//In case you are using postgreSQL jdbc driver.
conn = DriverManager.getConnection("jdbc:redshift://********url-to-redshift.redshift.amazonaws.com:5439/example-database", props);
System.out.println("\n\nConnection made!\n\n");
statement = conn.createStatement();
String command = "COPY my_table from 's3://path/to/csv/example.csv' CREDENTIALS 'aws_access_key_id=******;aws_secret_access_key=********' CSV DELIMITER ',' ignoreheader 1";
System.out.println("\n\nExecuting...\n\n");
statement.executeUpdate(command);
//you must need to commit, if you realy want to have data copied.
conn.commit();
System.out.println("\n\nThats all copy using simple JDBC.\n\n");
statement.close();
conn.close();
} catch (Exception ex) {
ex.printStackTrace();
}
}
}