I'm trying to consolidate multiple databases into a single database.
The challenge is, not all the databases have the same column structure.
In some cases, number of columns in the source table is very much less than the target table.
What I want to do is, map the columns that are common between source and target tables and set other columns in the target table to null.
e.g. Let's consider that my source table consists of 120 columns where as the target table consists of 180 columns. I want to map all the 120 columnsin the source table to corresponding column in the target table leaving the remaining 60 as null in the target.
Is it possible in Pentaho Kettle?
Currently I'm getting the following error:
2016/07/27 12:16:29 - backoffice1213 ClientAddMaster.0 - ERROR (version 6.1.0.1-196, build 1 from 2016-04-07 12.08.49 by buildguy) : Unexpected error
2016/07/27 12:16:29 - backoffice1213 ClientAddMaster.0 - ERROR (version 6.1.0.1-196, build 1 from 2016-04-07 12.08.49 by buildguy) : org.pentaho.di.core.exception.KettleStepException:
2016/07/27 12:16:29 - backoffice1213 ClientAddMaster.0 - Field [CustomerID] is required and couldn't be found!
2016/07/27 12:16:29 - backoffice1213 ClientAddMaster.0 -
2016/07/27 12:16:29 - backoffice1213 ClientAddMaster.0 - at org.pentaho.di.trans.steps.tableoutput.TableOutput.processRow(TableOutput.java:98)
2016/07/27 12:16:29 - backoffice1213 ClientAddMaster.0 - at org.pentaho.di.trans.step.RunThread.run(RunThread.java:62)
2016/07/27 12:16:29 - backoffice1213 ClientAddMaster.0 - at java.lang.Thread.run(Thread.java:745)
2016/07/27 12:16:29 - backoffice1112 ClientAddMaster.0 - ERROR (version 6.1.0.1-196, build 1 from 2016-04-07 12.08.49 by buildguy) : Unexpected error
2016/07/27 12:16:29 - backoffice1112 ClientAddMaster.0 - ERROR (version 6.1.0.1-196, build 1 from 2016-04-07 12.08.49 by buildguy) : org.pentaho.di.core.exception.KettleStepException:
2016/07/27 12:16:29 - backoffice1112 ClientAddMaster.0 - Field [CustomerID] is required and couldn't be found!
2016/07/27 12:16:29 - backoffice1112 ClientAddMaster.0 -
2016/07/27 12:16:29 - backoffice1112 ClientAddMaster.0 - at org.pentaho.di.trans.steps.tableoutput.TableOutput.processRow(TableOutput.java:98)
2016/07/27 12:16:29 - backoffice1112 ClientAddMaster.0 - at org.pentaho.di.trans.step.RunThread.run(RunThread.java:62)
2016/07/27 12:16:29 - backoffice1112 ClientAddMaster.0 - at java.lang.Thread.run(Thread.java:745)
2016/07/27 12:16:29 - backoffice1011 ClientAddMaster.0 - ERROR (version 6.1.0.1-196, build 1 from 2016-04-07 12.08.49 by buildguy) : Unexpected error
2016/07/27 12:16:29 - sohamdata1112 clientaddmaster.0 - Finished reading query, closing connection.
2016/07/27 12:16:29 - backoffice1011 ClientAddMaster.0 - ERROR (version 6.1.0.1-196, build 1 from 2016-04-07 12.08.49 by buildguy) : org.pentaho.di.core.exception.KettleStepException:
2016/07/27 12:16:29 - backoffice1011 ClientAddMaster.0 - Field [CustomerID] is required and couldn't be found!
2016/07/27 12:16:29 - backoffice1011 ClientAddMaster.0 -
2016/07/27 12:16:29 - backoffice1011 ClientAddMaster.0 - at org.pentaho.di.trans.steps.tableoutput.TableOutput.processRow(TableOutput.java:98)
2016/07/27 12:16:29 - backoffice1011 ClientAddMaster.0 - at org.pentaho.di.trans.step.RunThread.run(RunThread.java:62)
2016/07/27 12:16:29 - backoffice1011 ClientAddMaster.0 - at java.lang.Thread.run(Thread.java:745)
I've defined the mapping for common columns between the tables. Please let me know if any more info is needed.