Preparedbatch 方法采用大小为 1000 的结果集。我正在使用准备好的语句和批处理插入数据。
使用 LIMIT 我从 mysql 数据库、java 中提取 1000 行,并从每行的列值中计算一些值。
public class Method {
void preparedBatch (ResultSet rs, PreparedStatement insert) throws Exception{
int y;
int arr[]= new int[40];
while (rs.next()){
for(y=2;y<=41;y++){
arr[y-2]=rs.getInt(y);
}
insert.setString(1, rs.getString(1));
insert.setLong(2, calculate(arr));
insert.addBatch();
}
{
insert.executeBatch();
insert.clearBatch(); // I did this to avoid oOM
}
rs.close(); //if I do not close, then I get OOM error
}
long calculate (int[] arr){
long sum =0;
for(int k=0;k<40;k++){
sum+= arr[k]*pow(2,k);
}
return sum;
}
long pow(long base, int exponent)
{
if (exponent == 0)
return 1; // base case;
double temp = pow(base, exponent/2);
if (exponent % 2 == 0)
return (long) (temp * temp);
else
return (long) (base * temp * temp);
}
void method(){
Method k = Insert.m1;
System.out.println("in metho");
Database d1= new Database(); // for reading from tables
d1.setPassword("abc");
d1.setUrl("jdbc:mysql://localhost/DB");
d1.setUser("root");
// object to calculate 2 raise to 39 stuff
ResultSet rs =null;// result set for 2500 lines selected
PreparedStatement ps=null,insert=null; //ps for selecting 1000 lines //insert for inserting into binary table
final Connection con;
try {
con = DriverManager.getConnection(d1.getUrl(), d1.getUser(), d1.getPassword());
con.setAutoCommit(false);
insert = con.prepareStatement("insert into binarydata values(?,?)");
int z;
for(z=0;z<=850000;z=z+1000)
{
ps = con.prepareStatement("select * from table1 LIMIT "+z +", 1000");
rs= ps.executeQuery();
try {
k.preparedBatch(rs,insert);//k is global object of Methodclass
} catch (Exception e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
rs.close();
if(z%200000==0){
System.out.println(z +" inserted this time");
}
if(z==850000){
System.out.println(z +"mouse");
}
}
{
ps = con.prepareStatement("select * from table1 LIMIT 851000 , 364");
insert = con.prepareStatement("insert into binarydata values(?,?)");
rs= ps.executeQuery();
try {
k.preparedBatch(rs,insert);
System.out.println("finally inserting");
} catch (Exception e) {
// TODO Auto-generated catch block
e.printStackTrace();
System.out.println(rs.getString(1));
}
rs.close();
}
try {
con.commit();
} catch (SQLException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
finally{
con.close();
ps.close();
insert.close();
}
} catch (SQLException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
我的主类创建 Method 对象并从中调用方法,我的数组是 0,1,0,1,1,1,1,1,,0,0,0,0,0,0,0,1,0 (这是一个例子,所有 851364 行都包含这样的数据)
例如,如果我的 arr 是 0,1,0,那么我正在计算 sum+= arr[index]*2 (power index).. 像这样..
我不了解 Springs,否则我会用它来处理我大小的数据。
请帮助我理解,为什么在方法preparedBatch中,我必须关闭结果集?
另外,请建议优化代码。我花了大约 9 分钟,阅读 85136 行并插入相应的总和..