0

以下 ItemReader 获取数千个帐户 ( acc ) 的列表。

ItemReader 将连接到以检索数据的数据库是 HIVE。我没有创建任何表的权限，只有读取选项。

 @Bean
 @StepScope
public ItemReader<OmsDto> omsItemReader(@Value("#{stepExecutionContext[acc]}") List<String> accountList) {

    String inParams = String.join(",", accountList.stream().map(id -> 
"'"+id+"'").collect(Collectors.toList()));

    String query = String.format("SELECT ..... account IN (%s)", inParams); 
    
    BeanPropertyRowMapper<OmsDto> rowMapper = new BeanPropertyRowMapper<>(OmsDto.class);
    rowMapper.setPrimitivesDefaultedForNullValue(true);
    JdbcCursorItemReader<OmsDto> reader = new JdbcCursorItemReader<OmsDto>();

    reader.setVerifyCursorPosition(false);
    reader.setDataSource(hiveDataSource());
    reader.setRowMapper(rowMapper);
    reader.setSql(query);
    reader.open(new ExecutionContext());
    return reader;
}

这是我在使用 ItemReader 时收到的错误消息：

Caused by: org.springframework.batch.item.ItemStreamException: Failed to initialize the reader
at org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader.open(AbstractItemCountingItemStreamItemReader.java:153) ~[spring-batch-infrastructure-4.2.4.RELEASE.jar:4.2.4.RELEASE]

 Caused by: java.sql.SQLException: Error executing query
at com.facebook.presto.jdbc.PrestoStatement.internalExecute(PrestoStatement.java:279) ~[presto-jdbc-0.243.2.jar:0.243.2-128118e]
at com.facebook.presto.jdbc.PrestoStatement.execute(PrestoStatement.java:228) ~[presto-jdbc-0.243.2.jar:0.243.2-128118e]
at com.facebook.presto.jdbc.PrestoPreparedStatement.<init>(PrestoPreparedStatement.java:84) ~[presto-jdbc-0.243.2.jar:0.243.2-128118e]
at com.facebook.presto.jdbc.PrestoConnection.prepareStatement(PrestoConnection.java:130) ~[presto-jdbc-0.243.2.jar:0.243.2-128118e]
at com.facebook.presto.jdbc.PrestoConnection.prepareStatement(PrestoConnection.java:300) ~[presto-jdbc-0.243.2.jar:0.243.2-128118e]
at org.springframework.batch.item.database.JdbcCursorItemReader.openCursor(JdbcCursorItemReader.java:121) ~[spring-batch-infrastructure-4.2.4.RELEASE.jar:4.2.4.RELEASE]
... 63 common frames omitted
Caused by: java.lang.RuntimeException: Error fetching next at https://prestoanalytics-ch2-p.sys.comcast.net:6443/v1/statement/executing/20201118_131314_11079_v3w47/yf55745951e0beccc234c98f36005723457073854/0 returned an invalid response: JsonResponse{statusCode=502, statusMessage=Bad Gateway, headers={cache-control=[no-cache], content-length=[107], content-type=[text/html]}, hasValue=false} [Error: <html><body><h1>502 Bad Gateway</h1>
The server returned an invalid or incomplete response.
</body></html>
]

我确信根本原因是由于驱动程序，但这次我使用 DriverManager 使用相同的 SQL 测试了驱动程序，并且它运行完美。

 @Component
 public class OmsItemReader implements ItemReader<OmsDto>, StepExecutionListener {

private ItemReader<OmsDto> delegate;
public SikOmsItemReader() {
    
    Properties properties = new Properties();
    properties.setProperty("user", "....");
    properties.setProperty("password", "...");
    properties.setProperty("SSL", "true");
    
    Connection connection = null;
    
    try {
        connection = DriverManager.getConnection("jdbc:presto://.....", properties);
        Statement statement = connection.createStatement();
        
        ResultSet resultSet = statement.executeQuery(

我不确定有什么不同？是驱动程序还是备用批次？

我正在寻找一种解决方法。如何通过带有 spring batch 的 IN 子句检索数千个帐户？

谢谢

4

0 回答 0